July 2016 - dovecot - dovecot.org

Opportunistic quota recalc
by Tom Sommer 27 Jul '16

27 Jul '16

I'm switching quota backend to redis (from maildir++), so all my current usages are reset. I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash). It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login? Thanks. -- Tom Sommer

2 7

Dovecot v2.2.25 crashed on startup in Mac OSX
by Michael Chen 27 Jul '16

27 Jul '16

I built from dovecot-2.2.25.tar.gz on Mac OSX 10.11.15 (El Capitan) the usual way: CFLAGS=-I/usr/local/ssl/include LDFLAGS=-L/usr/local/ssl/lib ./configure make sudo make install sudo /usr/local/sbin/dovecot It consistently crash on startup with the following entries in /var/log/mail.log (edited backtrace): Jul 27 07:39:26 macbook-pro dovecot[70234]: master: Dovecot v2.2.25 (7be1766) starting up for imap, pop3, lmtp Jul 27 07:39:26 macbook-pro dovecot[70234]: master: Panic: kevent(EV_ADD, READ, 60) failed: Bad file descriptor Jul 27 07:39:26 macbook-pro dovecot[70234]: master: Error: Raw backtrace: 2 libdovecot.0.dylib 0x00000001000dc3fb default_fatal_finish + 43 -> 3 libdovecot.0.dylib 0x00000001000dd4e0 syslog_handler + 0 -> 4 dovecot 0x0000000100026f8d master_fatal_callback + 429 -> 5 libdovecot.0.dylib 0x00000001000dca9f i_panic + 415 -> 6 libdovecot.0.dylib 0x00000001000ff022 io_loop_handle_add + 194 -> 7 libdovecot.0.dylib 0x00000001000fb5f1 io_add_file + 385 -> 8 libdovecot.0.dylib 0x00000001000fb456 io_add + 118 -> 9 libdovecot.0.dylib 0x0000000100102b83 lib_signals_set_handler + 355 -> 10 dovecot 0x0000000100027070 main_init + 112 -> 11 dovecot 0x0000000100026628 main + 2024 -> 12 libdyld.dylib 0x00007fff889ac5ad start + 1 It looks similar to this report: http://article.gmane.org/gmane.mail.imap.dovecot/86319/match=panic+kevent HomeBrew installed 2.2.24, and it works in my Mac. Can someone confirm this is a bug? I also built 2.2.25 with debug, but can't make it break at line iolook-kqueue.c:67 i_panic("kevent(EV_ADD, READ, %d) failed: %m", io->fd); Thanks --mc . -- CONFIDENTIALITY NOTICE: This e-mail, including attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information or otherwise be protected by law. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender and destroy all copies and the original message.

2 2

Converting mailboxes to UTF-8
by Артем Перекресний 27 Jul '16

27 Jul '16

Is any proven routine to migrate mailboxes from mUTF-7 to UTF-8 encoding? I've try to use doveadm list/mutf7/rename but have been stuck with nested inboxes. Say I have the next structure: INBOX INBOX.жыр INBOX.жыр.ядъ encoded for FS respectively as .INBOX .INBOX.&BDYESwRA- .INBOX.&BDYESwRA-.&BE8ENARK- When I've try to rename INBOX.жыр with doveadm mailbox rename -u me -s "INBOX.&BDYESwRA-" $(doveadm mailbox mutf7 -7 "INBOX.&BDYESwRA-") I've get the next: .INBOX .INBOX.жыр .INBOX.жыр.&BE8ENARK- Sure when I've try to rename .INBOX.жыр.&BE8ENARK- I've get a message that this is not a valid mUTF7-encoded string. So that is the question.

2 3

imap & pop3 using same mailbox with virtual plugin
by Michael Fox 26 Jul '16

26 Jul '16

This is a noobie question, so bear with me if it's not worded correctly: Primary Requirements: 1) I'd like to have two namespaces - one private/per-user, and one public (visible by all logged-in users). The public mailbox would be used to distribute information to all users. 2) I'd like both POP3 and IMAP users to see both namespaces. As I understand it, this requires the use of the virtual plugin to create a virtual namespace which would mix together the private and public namespaces into one mailbox. I have read: http://wiki.dovecot.org/Plugins/Virtual https://sys4.de/de/blog/2013/02/11/dovecot-virtual-setup-mit-globaler-sieve- spamfilter-regel-fur-pop3-nutzer/ (via Google translation) I don't really understand the example MySQL code (too much "..." for me to follow) in the above web pages. As I understand it (and I could be very wrong), the solution relies on a MySQL query to determine which namespace should have inbox = yes -- the private namespace or the virtual namespace -- depending on if the user is an imap or pop3 user, respectively. Question 1: How would this be done on a user-by-user basis if the userdb is a passwd-file flat file? Would it be something like this? -- Userdb extra fields for IMAP User: userdb_namespace/inbox/inbox=yes -- Userdb extra fields for POP3 User: userdb_namespace/virtual/inbox=yes Question 2: If I want to allow any user to use either POP3 or IMAP, then could I do it this way? protocol imap { namespace inbox { inbox = yes } } protocol pop3 { namespace virtual { inbox = yes } } Thanks, Michael

1 0

Dovecot cannot find mailboxes
by C. Andrews Lavarre 26 Jul '16

26 Jul '16

Edgar, thank you for your help: I have solved gaining access from a client (Evolution) via IMAP to the server: I can refresh, add folders, and delete folders. But I am still confounded, as IMAP does not see any mailboxes. Neither we from the command line (telnet, openssl, login, select) nor the system can find the mailbox, despite following FindMailLocation: telnet, openssl, login, then b select inbox * 0 EXISTS * 0 RECENT and postfix fails with 2016-07-23T21:22:37.312039-04:00 lavarre postfix/error[17088]: A8DA2C1BB2: to=<andy(a)privustech.com>, orig_to=<andy>, relay But the mailboxes do exist in /var/mail/vhosts/privustech.com/andy and we point to them in /etc/dovecot/users. doveconf -n is attached. ----- Two issues are: 1. file permissions and ownership 2. virtual versus system users 1. It appears that having the correct ownership and permissions on various files is critical (of course), but those parameters must also be included in the interior of the /etc/dovecot/user. So if you change permissions you also must change that file. For example, you said: i also noticed your certificate chain is broken. http://wiki2.dovecot.org/testinstallation I worked through this link, thank you. Changing permissions for the directories /var/mail/* to root:mail ($UID:$GID = 1000:12) and then changing /etc/dovecot/users correspondingly to user(a)privustech.com :{plain}actualpassword: 1000:12 ::/var/mail/vhosts/privustech.com/user now allows a normal login. and we can create a new folder (e.g., work) and delete it. However, this is at variance with http://wiki2.dovecot.org/HowTo/SimpleVirtualInstall which says to set the ownership to vmail:vmail ($UID:$GID = 100: 5000). Doing so breaks dovecot. 2. The various links suggest that having virtual users is preferable to setting system users. The former do not require directories under /homebut they do require a directory in the mail system, which I have provided under /var/mail/vhosts. It also turns out that for virtual users you must include the domain (andy(a)privustech.com) Making that change allowed the success reported above. But despite that, we still cannot find the mailboxes on login. Any thoughts on how to have IMAP find the mailboxes (beyond FindMailLocation, which doesn't seem to work for me) would be most appreciated. Kind regards, Andy

3 2

[2.2.25 auth issue] auth-request.c: line 1049 (auth_request_lookup_credentials): assertion failed: (request->credentials_scheme == scheme)
by Alexey Asemov (Alex/AT) 26 Jul '16

26 Jul '16

Hello, Since updating to 2.2.25, we do have the following issue (seems to be related to the new auth policy code): Jul 9 00:00:14 LXmail01 dovecot: auth-worker(6558): sql(user(a)domain.org,*.*.*.*,<wjpa7SY3nNFR8yTo>): unknown user Jul 9 00:00:14 LXmail01 dovecot: auth: Panic: file auth-request.c: line 1049 (auth_request_lookup_credentials): assertion failed: (request->credentials_scheme == scheme) Jul 9 00:00:14 LXmail01 dovecot: auth: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x8d57e) [0x7f197bd8157e] -> /usr/lib64/dovecot/libdovecot.so.0(+0x8d65e) [0x7f197bd8165e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f197bd201f6] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](+0x1a15a) [0x7f197c44715a] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](+0x26f8f) [0x7f197c453f8f] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](+0x19d2b) [0x7f197c446d2b] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](auth_request_lookup_credentials_callback+0x68) [0x7f197c4 46e28] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](passdb_handle_credentials+0x92) [0x7f197c457742] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](+0x2af06) [0x7f197c457f06] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](+0x2065a) [0x7f197c44d65a] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f197bd94edc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_in ternal+0xff) [0x7f197bd9633f] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f197bd94f65] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f197bd95118] -> /usr /lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f197bd26663] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](main+0x3a1) [0x7f197c43cbe1] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f197b30ab15] -> dovecot/auth [1 wait, 0 passdb, 0 userdb](+0xfdd1) [0x7f197c43cdd1] Jul 9 00:00:14 LXmail01 dovecot: auth: Fatal: master: service(auth): child 6555 killed with signal 6 (core dumps disabled) Jul 9 00:00:14 LXmail01 dovecot: pop3-login: Warning: Auth connection closed with 1 pending requests (max 1 secs, pid=8133, EOF) Environment is MySQL passdb/userdb, we are using plain scheme as default and password_noscheme field. In 2.2.24 and below auth worked without scratch. Reverting all policy-related commits in 2.2.25 fixes this, but I have not dug the details. KR, Alex

3 2

Doveadm error
by László Károlyi 25 Jul '16

25 Jul '16

Hey everyone, I've got a weird error since I upgraded to the latest dovecot on my FreeBSD box: root@postfixjail /# doveadm quota recalc -u xxx(a)xxx.com doveadm(xxx(a)xxx.com): Error: dict-client: Commit failed: Deinit fish: 'doveadm quota recalc -u xxx@…' terminated by signal SIGSEGV (Address boundary error) root@postfixjail /# dovecot --version 2.2.25 (7be1766) root@postfixjail /# dovecot -n # 2.2.25 (7be1766): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: FreeBSD 10.3-STABLE amd64 auth_cache_negative_ttl = 0 auth_cache_ttl = 0 auth_mechanisms = plain login cram-md5 digest-md5 auth_realms = flix.hu base_dir = /usr/local/var/run/dovecot/ default_login_user = nobody dict { quota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext quota_rule2 = Trash:bytes=+100M } listen = * login_trusted_networks = 127.0.0.0/24 mail_location = mdbox:~/mdbox mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe mdbox_rotate_size = 20 M namespace { inbox = yes location = prefix = separator = . } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /usr/local/etc/email-responder sieve_plugins = sieve_extprograms } postmaster_address = postmaster(a)flix.hu protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { process_min_avail = 3 service_count = 1 } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 2 service_count = 1 } service managesieve { process_limit = 1024 process_min_avail = 2 } service pop3-login { process_min_avail = 3 service_count = 1 } ssl = required ssl_cert = </usr/local/etc/dovecot/ssl/server.crt ssl_key = </usr/local/etc/dovecot/ssl/server.pem userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota sieve } protocol lmtp { mail_plugins = quota sieve } protocol sieve { mail_plugins = quota sieve managesieve_max_line_length = 65536 } protocol imap { mail_max_userip_connections = 20 mail_plugins = quota imap_quota } protocol pop3 { mail_max_userip_connections = 15 mail_plugins = quota } root@postfixjail /# uname -a FreeBSD postfixjail.xxx.com 10.3-STABLE FreeBSD 10.3-STABLE #19 r302639: Tue Jul 12 13:54:21 CEST 2016 root@flix.hu:/usr/obj/usr/src/sys/MYKERNEL amd64 There seems no difference of which virtual mailbox I want dovecot to quota recalculate, it always fails with this error message. Any suggestions? Cheers, -- László Károlyi http://linkedin.com/in/karolyi

5 6

maildirsize calculation gives higher value than a real disk utilization
by Andrea Lisci 25 Jul '16

25 Jul '16

Hi, hope someone can help also if this problem is difficult to debug I have a mail account that has on file system 32GB however quotacalculation gives a higher value 41262146 I've already tried to: - doveadm quota recalc -u user@domain - removing maildirsize from /users/vmail/domain/user/Maildir - checking all emails messages if they have the size in the name (S=xxxx) different from the real size. I used the script here in attach giving it in input the output of << find /users/vmail/domain/user -name *S=* -type f>> I don't have mail compression the mail account have a lot of mailboxes and at now 78319 messages doveadm quota get -u user@domain Quota name Type Value Limit % User quota STORAGE 41262146 42991616 95 User quota MESSAGE 78319 - 0 this is the same command run with -Dv doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so doveadm(root): Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_backend_rescan (this is usually intentional, so just ignore this message) doveadm(user@domain): Debug: Added userdb setting: mail_location=maildir:/users/vmail/domain/user/Maildir doveadm(user@domain): Debug: Added userdb setting: plugin/quota_rule=*:storage=41G doveadm(user@domain): Debug: Effective uid=5000, gid=5000, home=/users/vmail/domain/user doveadm(user@domain): Debug: Quota root: name=User quota backend=maildir args= doveadm(user@domain): Debug: Quota rule: root=User quota mailbox=* bytes=44023414784 messages=0 doveadm(user@domain): Debug: Quota warning: bytes=41822244044 (95%) messages=0 reverse=no command=quota-warning 95 user@domain doveadm(user@domain): Debug: Quota grace: root=User quota bytes=20971520 doveadm(user@domain): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/users/vmail/domain/user/Maildir doveadm(user@domain): Debug: maildir++: root=/users/vmail/domain/user/Maildir, index=, indexpvt=, control=, inbox=/users/vmail/domain/user/Maildir, alt= Quota name Type Value Limit % User quota STORAGE 41202926 42991616 95 User quota MESSAGE 78247 - hope someone can help I'm running dovecot-2.2.10-5.el7 on centos7

3 3

postconf -n display error for "remote"?
by Michael Fox 25 Jul '16

25 Jul '16

Is this a bug? I configured the following: remote 192.168.7.128/27 { ssl = no } remote 192.168.7.0/24 { ssl = yes } But, when I run doveconf -n, I see: remote 192.168.7.0/24/24 { ssl = yes } remote 192.168.7.128/27/27 { ssl = no } Note the repeated network prefix length (/27/27 and /24/24). Is there a way to tell for sure whether this is just a cosmetic issue or something that could cause a problem? Michael

2 1

Cannot connect to Dovecot IMAP or POP
by C. Andrews Lavarre 24 Jul '16

24 Jul '16

Hello all. Thank you for your service. Easy when you know how, but presently I do not. After literally months of research and experimentation we simply cannot log into our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email server with an ordinary email client, e.g., Evolution or Thunderbird. We can connect to the host server in a host of different ways (no pun intended)—http, https, ssh, vnc, telnet, openssl -sclient Similarly we can connect to postfix and dovecot in yet another number of ways—telnet, openssl -sclient—but cannot log in to the email server with a normal email client (either Evolution or Thunderbird) by either pop3 or imap. SSL certificates are in place, verified, and tested. Part of the problem is the many changes in all the involved operating systems and protocols (e.g., imaps and pop3s are deprecated, openSUSE has migrated to LEAP, etc.) so many of the docs from Google are no longer valid. Additionally, there simply are bugs: Leap 42.1 YAST does not work when it comes to setting up websites. Documented. But I digress. I'm sure it's something really simple, but it evades me. Research details below. Any help would be more than appreciated. Thanks in advance, Andy ======================= Configuration testing details ======================= System is: Linux openSUSE Leap 42.1 Dovecot --version 2.2.18, Postfix Version: 2.11.6-3.1 Apache2 Version: 2.4.16-9.1 Connections 1. Evolution or Thunderbird to pop3 or imap reports: The reported error was "Could not connect to mail.privustech.com: Connection refused". Both connect successfully to googlemail.com with the same protocol: Port 993 SSL on a dedicated port I have also tried Port 143 STARTTLS after connecting without success 2. openssl s_client -connect mail.privustech.com:xxx a. xxx=25, 110, 143 all return error:140770FC b. xxx=993, 995 return socket: Connection refused connect:errno=111 3.telnet to a. smtp works. b. pop3 andy@tm2t:~> telnet 70.186.159.22 110 ... +OK POP3 2007e.104 server ready <48fa.572a0769(a)privustech.com> ... user andy -ERR Unknown AUTHORIZATION state command c. imap connects but does not allow login, and should not. http://marc.info/?l=imap&m=118775891829506&w=2 The most simple answer is "you cannot TELNET to a modern, correctly-configured, IMAP server and log in to it." andy@tm2t:~> telnet 70.186.159.22 143 ... * OK [...] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 10:26:28 -0400 (EDT) ... A NO Invalid login credentials Modules • Apache2 works just fine. The server is up and answering. ping works just fine. We have http and https to all vhost sites (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and their www. subsites). • Postfix reports no errors. We can log in on localhost, send a message to ourselves and see the message. • Dovecot: a. Logging is enabled in 10-logging.conf to /var/log/dovecot.conf but no logging has occurred there. b. doveconf -n throws no errors. Checks and tests completed 1. /etc/hosts is just fine. 2. Firewall is open for telnet, postfix, dovecot. 3. Added andy to dovecot, postfix groups, in addition to mail, reset password to ANDYbbs14@. 4. We tried enabling imaps, pop3s, but this command returns errors about these protocols being obsolete. https://tools.ietf.org/html/rfc2595 Use of these ports is discouraged in favor of the STARTTLS or STLS commands. 5. Reviewed doveconf -n: a. Note, there are no Dovecot users established other than user postfix group postfix service auth { unix_listener auth-userdb { group = postfix user = postfix } } i. postfix has its own set of users, including andy, which works just fine within postfix. We can send mail and read mail in the mailbox. b. Authentication is performed by PAM: passdb { driver = pam } i. Examined PAM: A. The files /etc/pam.d/xxx, where xxx = dovecot, pop, imap, are all the same lavarre:~ # cat /etc/pam.d/xxx #%PAM-1.0 auth include common-auth account include common-account password include common-password session include common-session B. They do not resemble at all the form presented in http://wiki2.dovecot.org/PasswordDatabase/PAM passdb { driver = pam args = %s } C. Add (B.) to see if that works: No change. Comment out the original (A.): No change. Restore it. c. SSL is required and apparently configured correctly (the less-than symbol '<'causes the succeeding file to be read into the variable): ssl = required ssl_cert = </etc/ssl/certs/dovecot.pem ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/dovecot.pem ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } i. dovecot.pem, both cert and key, are installed in /etc/ssl as above and verified as a pair with openssl x509. And we point to them in /etc/dovecot/conf.d/10-ssl.conf as seen in the above. 6. Checked listening as it does not appear in doveconf -n: lavarre:~ # doveconf protocols listen protocols = imap pop3 lmtp listen = *, :: a. conf.d/10-master.conf ports for service xxx-login {inet_listener} are commented out. In fact, the entire file is commented out. Uncomment the listeners, restart. But no change. So undo.

2 3