February 2016 - dovecot

[BUG] Dovecot 2.2.21 crashing with assertion failed: (!have_wait_unfinished || unfinished_count > 0)
by Gary Stanley 12 Feb '16

12 Feb '16

Hello Dovecot's IMAP process appears to be crashing at random times for a few users, but one is crashing at least 3 to 5 times an hour. I applied these patches per http://dovecot.org/list/dovecot/2016-January/102933.html /https://github.com/dovecot/core/commit/1ddf959a750f3860feff4ab3f0e908f32740978f.diff //https://github.com/dovecot/core/commit/c8e9fa2ffa2566e75f0500808b1bc9bf5d9db024.diff //https://github.com/dovecot/core/commit/15307c2c91854e766bd9fb095d611a29b3f7572e.diff //https://github.com/dovecot/core/commit/c7801f830c7d2e7d340065cdd5a5c795b1726223.diff //https://github.com/dovecot/core/commit/f136b0050b3125b466af73984177250b7ed1a3be.diff / But it's still crashing. # 2.2.21 (5345f22): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-604.30.3.lve1.3.63.el6.x86_64 x86_64 CloudLinux Server release 6.7 (Aleksei Gubarev) auth_cache_size = 1 k auth_mechanisms = plain login auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$-=?^_{}~./@+%" disable_plaintext_auth = no first_valid_uid = 201 mail_plugins = " zlib" maildir_very_dirty_syncs = yes namespace { inbox = yes location = mailbox Archive { auto = create special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = create special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. type = private } passdb { args = /usr/local/cpanel/bin/dovecot-wrap driver = checkpassword } plugin { acl = vfile quota = maildir quota_rule = INBOX.Trash:ignore } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0666 } } service imap-login { client_limit = 500 inet_listener imap { address = * } inet_listener imaps { address = * } process_limit = 50 process_min_avail = 2 service_count = 0 vsz_limit = 64 M } service imap { process_limit = 512 vsz_limit = 256 M } service managesieve-login { client_limit = 500 process_limit = 50 process_min_avail = 2 service_count = 0 vsz_limit = 64 M } service managesieve { process_limit = 512 vsz_limit = 256 M } service pop3-login { client_limit = 500 inet_listener pop3 { address = * } inet_listener pop3s { address = * } process_limit = 50 process_min_avail = 2 service_count = 0 vsz_limit = 64 M } service pop3 { process_limit = 512 vsz_limit = 256 M } ssl_cert = </etc/dovecot/ssl/dovecot.crt ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_key = </etc/dovecot/ssl/dovecot.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } userdb { args = /usr/local/cpanel/bin/dovecot-wrap driver = checkpassword } protocol imap { imap_capability = +NAMESPACE imap_idle_notify_interval = 24 mins imap_logout_format = in=%i, out=%o, bytes=%i/%o mail_max_userip_connections = 20 mail_plugins = acl quota imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, bytes=%i/%o pop3_uidl_format = UID%u-%v } protocol lda { postmaster_address = postmaster(a)example.com } 1 18:23:54 gw-ud3 dovecot: imap(xxxx(a)xxxx.com): Panic: file imap-client.c: line 849 (client_check_command_hangs): assertion failed: (!have_wait_unfinished || unfinished_count > 0) Feb 11 18:23:54 gw-ud3 dovecot: imap(xxxx(a)xxxx.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x8096a) [0x7f996c47e96a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x809d6) [0x7f996c47e9d6]$ Feb 11 18:23:55 gw-ud3 dovecot: imap(xxxx(a)xxxx.com): Fatal: master: service(imap): child 329519 killed with signal 6 (core dumped) Here's the backtrace: (gdb) bt full #0 0x00007f046eb2e625 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 resultvar = 0 pid = 98454 selftid = 98454 #1 0x00007f046eb2fe05 in abort () at abort.c:92 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x2, sa_sigaction = 0x2}, sa_mask = {__val = {139657018134932, 1, 18488448, 0, 139657017886537, 139657017552035, 139657017530872, 206158430224, 140725557583888, 140725557583680, 4362456, 140725557583788, 18440536, 0, 0, 6717455502603727992}}, sa_flags = 1635021663, sa_restorer = 0x7ffd38dee060} sigs = {__val = {32, 0 <repeats 15 times>}} #2 0x00007f046ef10a60 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:201 backtrace = 0x1196158 "/usr/lib64/dovecot/libdovecot.so.0(+0x80a6a) [0x7f046ef10a6a] -> /usr/lib64/dovecot/libdovecot.so.0(+0x80ad6) [0x7f046ef10ad6] -> /usr/lib64/dovecot/libdovecot.so.0(i_error+0) [0x7f046eeb994f] -> dove"... #3 0x00007f046ef10ad6 in i_internal_fatal_handler (ctx=0x7ffd38dee080, format=<value optimized out>, args=<value optimized out>) at failures.c:670 status = 0 #4 0x00007f046eeb994f in i_panic (format=0x18096 <Address 0x18096 out of bounds>) at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffd38dee160, reg_save_area = 0x7ffd38dee0a0}} #5 0x0000000000418247 in client_continue_pending_input () No symbol table info available. #6 0x00007f046ef22fdb in io_loop_call_io (io=0x11c5860) at ioloop.c:559 ioloop = 0x119e740 t_id = 2 __FUNCTION__ = "io_loop_call_io" #7 0x00007f046ef24577 in io_loop_handler_run_internal (ioloop=<value optimized out>) at ioloop-epoll.c:220 ctx = 0x11a0250 events = <value optimized out> event = 0x11a10c0 list = 0x11a1c80 io = <value optimized out> tv = {tv_sec = 29, tv_usec = 999480} events_count = <value optimized out> msecs = <value optimized out> ret = 1 i = <value optimized out> call = <value optimized out> ---Type <return> to continue, or q <return> to quit--- __FUNCTION__ = "io_loop_handler_run_internal" #8 0x00007f046ef23085 in io_loop_handler_run (ioloop=0x119e740) at ioloop.c:607 No locals. #9 0x00007f046ef23228 in io_loop_run (ioloop=0x119e740) at ioloop.c:583 __FUNCTION__ = "io_loop_run" #10 0x00007f046eebeca3 in master_service_run (service=0x119e5e0, callback=<value optimized out>) at master-service.c:640 No locals. #11 0x0000000000424c5c in main () No symbol table info available.

2 2

Re: Question on setting up dovecot
by Joseph Tam 12 Feb '16

12 Feb '16

Jeffery Small writes: > There are no errors. We have a pop3 connection. The user's name, > uid, gid, password, home directory are all correct. The inbox is > listed as /var/mail/sherril which is correct, and this file has the > correct ownership and permissions: > > # ls -l /var/mail/sherril > -rw-rw---- 1 sherril mail 194900 Feb 11 09:00 /var/mail/sherril > > Yet, no mail is pulled over. Are there any clues here? OK, the next step might be to try talk POP3 directly to your dovecot server, and see if it responds to some common POP3 requests. You might have to use one of telnet server 110 nc server 110 (or "ncat" or "netcat") openssl s_client -connect server:995 (if using SSL) then test with these commands USER someuser PASS their-password (Should list messages and their sizes) LIST (Should list messages and their UIDLs) UIDL (Retrieve the first message) RETR 1 QUIT If these work, then I would look at the client. If not, check the logs why it failed. > Feb 11 08:47:49 pop3(sherril): Info: Disconnected: Logged out top=0/0, \ > retr=0/0, del=0/3, size=36928 If I read this right, the client never tried to download any messages (no top, no retr). Maybe the client downloaded the UIDLs and figured it already had local copies of those messages. Joseph Tam <jtam.home(a)gmail.com>

1 0

Config file syntax in gory detail
by Timo Sirainen 11 Feb '16

11 Feb '16

I finally wrote this: http://wiki2.dovecot.org/ConfigFile Looks like it ended up having a lot more content than I initially assumed.

5 4

Cache unseen count
by Jorge Carling 11 Feb '16

11 Feb '16

Dear all, Lately I am working on PHP mail account management application. It has, among others, Unread messages count column. To get the number of unread messages, I execute "doveadm search" command for each database row. The search time for single account is about 0.03 seconds. Our system has around 750 accounts, so the total loading time is currently around 20 seconds. When setting up Dovecot, I made the user_query, which is also used to store the message quota, and total number of messages stored in particular mailbox. Is there a keyword that can be included in the user_query, or some other way to maintain the unread messages count in the database? Maybe there is a way to execute a script every time a message is received, or becomes read, or unread. Kind regards,JC

2 1

Help migrating a mailbox...
by Willy Gardiol 11 Feb '16

11 Feb '16

Hi! i have a question for you guys, i have been playing with this issue for a while and i am a bit lost. I have dovecot 2.2.16 on a Gentoo linux server, which run a couple of email domains, linked with postfix. I have a huge (>4gb) Outlook PST file which contains a backup coming from a new user, he used to have this backup on his old email system and i need to dump it into his new email account on my server. This is just a "old" backup which needs to be kept for reference, there is no need to preserve any ID/UID/GUID whatever. So, i found the "readpst" util which is capable ot reading the PST file and dump it's contents as mbox structure. (see http://bit-vonameln.de/email-migration-outlook-dovecot-imap/) I did it and i got a 4.4gb folder with some sub-folders, attachments, and more files inside. Then i spent most of a few days trying to have dovecot digest this and import it on top of the user new email address. With no luck. I have being trying with doveadm sync but i cannot find how i need to do it. Note that: 1) there is no local user, only the mail user (user(a)hisdomain.org) 2) i use maildir, the PST has been converted to mbox 3) dovecot is compiled with maildir and mbox support I tried, for example: server # export mail_location=mbox:/path/to/mbox/converted/emails server # doveadm sync -u user(a)hisdomain.org -d doveadm(user(a)hisdomain.org): Fatal: User has no mail_replica in userdb Then i tried: server # export mail_location=/home/vmail/hisdomain.org/user/maildir server # doveadm sync -u user(a)hisdomain.org mbox:/path/to/mbox/converted/emails dsync(user(a)hisdomain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) Also tried: server # export mail_location=/home/vmail/hisdomain.org/user/maildir server # doveadm sync -u user(a)hisdomain.org /path/to/mbox/converted/emails doveadm(user(a)hisdomain.org): Fatal: execvp(/home/vmail/hisdomain.org/user/maildir) failed: Permission denied dsync-local(user(a)hisdomain.org): Error: read(remote) failed: EOF (version not received) My dovecot is configured to use "maildir", the most relevant part of my dovecot.conf is: auth_debug = no login_greeting = Dovecot ready. mail_debug = no auth_debug_passwords = no auth_default_realm = mydomain.org auth_mechanisms = plain login auth_realms = <list of my domains here> auth_verbose = no base_dir = /var/run/dovecot/ disable_plaintext_auth = no mail_uid=999 mail_gid=999 listen = * mail_location = maildir:/home/vmail/%d/%n/maildir protocols = imap sieve lmtp (i can add omitted lines if needed) either i am not undesrtanding anything, which is totally possible, or i am missing something big. If anybody on this list has any insight or suggestion, pelase advise! ps: the destination mailbox is perfectly new and empty, it's already existing and the user can successfully login, send and receive emails. Dovecot works great, and i am operating locally on the server where i uploaded the converted mbox folders. -- Willy Gardiol willy(a)gardiol.org www.gardiol.org www.trackaway.org -> Track YOUR way the way you want!

1 0

Pigeonhole Sieve fails on date-based filter rules - bug?
by Clemens Ender 10 Feb '16

10 Feb '16

Hi list! I’m running dovecot + pigeonhole on a debian wheezy based VPS. Here is the output of „dovecot -n“ # 2.2.18: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.9 disable_plaintext_auth = no imap_idle_notify_interval = 29 mins mail_plugins = acl quota fts fts_lucene managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace { location = maildir:%%h/maildir:INDEX=~/maildir/shared/%%u prefix = shared/%%u/ separator = / type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = username_format=%u /home/dovecot/auth.d/%d/users driver = passwd-file } plugin { acl_shared_dict = file:/home/dovecot/vmail/%d/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_decoder = decode2text fts_lucene = whitespace_chars=@ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 } user = dovecot } service lmtp { executable = lmtp -L unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/certs/dovecot.crt ssl_key = </etc/ssl/private/dovecot.key userdb { args = username_format=%u /home/dovecot/auth.d/%d/users default_fields = uid=vmail gid=vmail home=/home/dovecot/vmail/%d/%n userdb_mail=maildir:~/maildir quota_rule=*:storage=2G driver = passwd-file } protocol lmtp { mail_plugins = acl quota fts fts_lucene sieve postmaster_address = postmaster(a)en-der.at } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log mail_plugins = acl quota fts fts_lucene sieve } I’m trying to filter emails based on the following rule: if allof (date :value "lt" "Date" "date" "2015-01-01") { fileinto "Archiv/2014"; } Some mails which should match are not filtered. Please look at what sieve-filter -e -W -v ... puts out: >> Filtering message: ID: <0.0.6.888.1CFAFF845D5E634.77992(a)mta144.avaaz.org> Date: Size: 17781 bytes Subject: =?utf-8?q?Klimafreunde_-_der_Plan_ste... Performed actions: * store message in folder: Archiv/2014 Implicit keep: (none) >> Filtering message: ID: <1527093595.86163601420098718229.JavaMail.email(a)email.apple.com> Date: Thu, 1 Jan 2015 07:51:58 +0000 (GMT) Size: 19573 bytes Subject: =?ISO-8859-1?Q?Ihre_Rechnung,_Nr.=A01... Performed actions: (none) Implicit keep: * store message in folder: INBOX ... So it appears that the „Date“ header is not properly extracted for the affected emails. Here are the headers of an particular email (the one without „Date:“ listed above): Return-Path: <bounces-43375-33947798(a)bounces.avaaz.org> Delivered-To: <clemens(a)en-der.at> Received: from vps1.en-der.at by vps1 (Dovecot) with LMTP id Vrb9JNSl31M5aQAAwqatwA for <clemens(a)en-der.at>; Mon, 04 Aug 2014 15:25:08 +0000 Received: from mta144.avaaz.org (mta144.avaaz.org [69.60.9.144]) by vps1.en-der.at (Postfix) with ESMTP id 61715ACDAA for <clemens(a)en-der.at>; Mon, 4 Aug 2014 15:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=av_key; d=avaaz.org; h=Content-Type:MIME-Version:Subject:From:To:List-Unsubscribe:Message-ID; i=avaaz(a)avaaz.org; bh=aNrRG1oOw57rU4AT/0miGg8aWAI=; b=erOsRngaWCaW7R0sK8COa4k97QpCISFa6/0ZKfZJHAEHKJu3FbUTExJb/L6OQez/V26UgwpbKJqU Py0l8dDpu68tUlhR42AUnZV+M+lpqI6LdoiTt/QMC0TklQnqdKZt0E4CG+Kz7plluYwhXlZHVSXc +apdPqllFZeYAcsaN5k= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=av_key; d=avaaz.org; b=YdIs06lkIN6pxGKoLORIDjbBCLU/5vwP1QCVocIa0qB4EgT1uKwf+bDlpbQVWmSsjPzekeZmWbpZ um+paKzqq7UGtJMVb1eGjod/bkuOGeXXdeZG0tGJz8wfLdNog6DbfChRBvMciS/Wsz+xrK3yVbX6 bQuKbeD/gurWPhEDfFc=; Received: by mta144.avaaz.org id hruita1u62ku for <clemens(a)en-der.at>; Mon, 4 Aug 2014 11:25:07 -0400 (envelope-from <bounces-43375-33947798(a)bounces.avaaz.org>) Content-Type: multipart/alternative; boundary="===============1522543492113580631==" MIME-Version: 1.0 Subject: =?utf-8?q?Klimafreunde_-_der_Plan_steht!?= From: Ricken Patel - Avaaz.org <avaaz(a)avaaz.org> To: "clemens(a)en-der.at" <clemens(a)en-der.at> List-Unsubscribe: <https://secure.avaaz.org/act/?r=unsub&cl=5597832458&email=clemens@en-der.at…> X-Campaign-ID: avaazCLIMATEeventhostsrecruitment201407301851073204 X-Version-ID: 43375 Message-ID: <0.0.6.888.1CFAFF845D5E634.77992(a)mta144.avaaz.org> --===============1522543492113580631== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Liebe alle, tausende von uns haben ... Any idea what is wrong here? Best regards, Clemens Ender

1 0

Re: Question on setting up dovecot
by Joseph Tam 10 Feb '16

10 Feb '16

Jeffery Small writes: > I'm new to dovecot and have a configuration question. I'm attempting to > have a Windows 7 system on my secure LAN connect to the Ubuntu server > using POP3 on channel 110 and retrieve mail from the /var/mail directory. > The user is defined on the Ubuntu machine and when running the Windows > Internet E-Mail Settings tool, the Test Account Settings Logs onto the > server successfully and sends a test message. Sending message is a distinct service from retrieving Email. > However, when I perform a > Send/Receive operation from the user's Outlook, all steps complete but no > mail is pulled down from the server. > > I have tried setting: mail_location = /var/mail/%u > > and restarted dovecot, but nothing changes. How can I debug this? I'm > unsure where to look for clues as to what's happening under the hood. Any > pointers would be appreciated. Log files would be a good start. Maybe start with something like log_path = /path/to/logfile auth_verbose = yes auth_debug = yes mail_debug = yes verbose_ssl = yes Joseph Tam <jtam.home(a)gmail.com>

1 0

user password in mail-filter plugin
by Thomas Möhle 10 Feb '16

10 Feb '16

Hello, I would like to use the users password within a mail-filter script. Dovecots %w variable is only available within dovecot-auth, is there any way to access the password for a mail-filter? Thanks, Thomas

4 11

Sieve question
by ＠lbutlr 10 Feb '16

10 Feb '16

Is it possible to use redirect in a sieve to redirect to multiple addresses? if header :contains “from” “test(a)example.xn--com-9o0a { redirect “user1@example.com,user2@example.com”; } I thought this would work, But I am seeing a “redirect address is invalid: not a single addres [sic] (found ‘,’). -- I'll have what the gentleman on the floor is having.

3 3

Question on setting up dovecot
by Jeffery Small 10 Feb '16

10 Feb '16

Server OS: Ubuntu 15.10 Sendmail: 8.14.9-4 # doveconf -n ----------------------------------------------------------- # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 4.2.0-27-generic x86_64 Ubuntu 15.10 disable_plaintext_auth = no mail_location = mbox:~/lib/Mail:INBOX=/var/mail/%u passdb { driver = pam } protocols = imap pop3 ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { driver = passwd } userdb { driver = passwd } ----------------------------------------------------------- I'm new to dovecot and have a configuration question. I'm attempting to have a Windows 7 system on my secure LAN connect to the Ubuntu server using POP3 on channel 110 and retrieve mail from the /var/mail directory. The user is defined on the Ubuntu machine and when running the Windows Internet E-Mail Settings tool, the Test Account Settings Logs onto the server successfully and sends a test message. However, when I perform a Send/Receive operation from the user's Outlook, all steps complete but no mail is pulled down from the server. I have tried setting: mail_location = /var/mail/%u and restarted dovecot, but nothing changes. How can I debug this? I'm unsure where to look for clues as to what's happening under the hood. Any pointers would be appreciated. Regards, -- Jeff Small

1 0