- dovecot - dovecot.org

"Timeout (180s) while waiting for lock for transaction log file" since 2.3 upgrade?
by Zube 24 Jan '18

24 Jan '18

dovecot 2.1.x and 2.2.x has "just worked" with a minimal configuration for just about anything I've ever thrown at it. Now, with 2.3.0, there is one user (and only one user) that continually sees horrible delays, which is not surprising given the logs: Error: imap(user)<70175><356C8oljxrCBUppP>: Timeout (180s) while waiting for lock for transaction log file /path/to/home/directory/.imap/dovecot.list.index.log (WRITE lock held by pid 69995) where 69995 is the pid of the first dovecot process and 70175 is the pid of, e.g., the first LIST command. What's unique about this user is that he's the only one using linux (thunderbird) and his home directory is NFS-mounted. I understand that something is incorrect with the locking, but there are a couple of issues that I do not understand: 1) Is there a default that changed between 2.2 and 2.3 that would cause this? Despite potential issues associated with NFS, I didn't see the problem with 2.2. There is nothing in my conf file related to locking, so if there was a change, it must be a default for 2.3 (from doveconf -a): dotlock_use_excl = yes lock_method = fcntl mbox_dotlock_change_timeout = 2 mins mbox_lock_timeout = 5 mins mbox_read_locks = fcntl mbox_write_locks = dotlock fcntl 2) If the problem is related to NFS locks, why would the problem persist when the user runs thunderbird from the server where his home directory *is not* NFS-mounted? Thanks for any help. Cheers, Zube

1 0

Autoexpunge is not working with dotlock locking
by Jaakko Seppälä 24 Jan '18

24 Jan '18

Hello, Autoexpunge is not working when used with dotlock locking. It seems to been broken since version 2.2.28. When autoexpunge is enabled and mailbox locking is set to dotlock all imap, pop and lmtp connections crash with the same error "file file-lock.c: line 287: unreached". Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN] Dovecot ready. X LOGIN testuser password X OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE LITERAL+ NOTIFY] Logged in X LOGOUT * BYE Logging out X OK Logout completed (0.001 + 0.000 secs). Connection closed by foreign host. => Jan 22 09:25:55 dovecottest dovecot: imap-login: Login: user=<testuser>, method=PLAIN, rip=::1, lip=::1, mpid=8274, secured, session=<EDVCNV5jvNsAAAAAAAAAAAAAAAAAAAAB> Jan 22 09:25:57 dovecottest dovecot: imap(testuser)<8274><EDVCNV5jvNsAAAAAAAAAAAAAAAAAAAAB>: Panic: file file-lock.c: line 287: unreached Jan 22 09:25:57 dovecottest dovecot: imap(testuser)<8274><EDVCNV5jvNsAAAAAAAAAAAAAAAAAAAAB>: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0xc8d24) [0x7ff43bef5d24] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xc8dde) [0x7ff43bef5dde] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ff43be69190] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xcd8ff) [0x7ff43befa8ff] -> /usr/local/lib/dovecot/libdovecot.so.0(file_wait_lock_error+0x26) [0x7ff43befac46] -> /usr/local/lib/dovecot/libdovecot.so.0(file_try_lock_error+0x13) [0x7ff43befacc3] -> /usr/local/lib/dovecot/libdovecot.so.0(file_create_locked+0x253) [0x7ff43bef7df3] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_user_lock_file_create+0xc5) [0x7ff43c1f5f15] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_user_autoexpunge+0x1f8) [0x7ff43c1dfc68] -> dovecot/imap(+0x1ba7c) [0x7ff43c72fa7c] -> dovecot/imap(client_input+0xb5) [0x7ff43c7308c5] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x65) [0x7ff43bf0d315] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7ff43bf0ebbf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x52) [0x7ff43bf0d412] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ff43bf0d638] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ff43be8bb73] -> dovecot/imap(main+0x333) [0x7ff43c7226c3] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff43ba8bc05] -> dovecot/imap(+0xe8a1) [0x7ff43c7228a1] Jan 22 09:25:57 dovecottest dovecot: imap(testuser)<8274><EDVCNV5jvNsAAAAAAAAAAAAAAAAAAAAB>: Fatal: master: service(imap): child 8274 killed with signal 6 (core dumps disabled) Configuration used for testing: [root@dovecottest dovecot]# /usr/local/sbin/dovecot -n # 2.3.0 (c8b89eb): /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.el7.x86_64 x86_64 CentOS Linux release 7.3.1611 (Core) lock_method = dotlock mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { autoexpunge = 30 days } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } service lmtp { inet_listener lmtp { port = 24 } } ssl = no userdb { driver = passwd } Regards, Jaakko

2 1

version conflict in freebsdports?
by ＠lbutlr 24 Jan '18

24 Jan '18

I have dovecot 2.2.33 installed. When trying to install dovecot2-pigeonhole (v 0.4.19) I get the following when it tries to install dovecot 2.2.31_1 I run portsnap fetch update daily, and I ran it again after this error, but same result. ===> Installing for dovecot2-2.2.31_1 ===> Checking if dovecot2 already installed ===> Registering installation for dovecot2-2.2.31_1 as automatic Installing dovecot2-2.2.31_1... pkg-static: dovecot2-2.2.31_1 conflicts with dovecot-2.2.33.2_2 (installs files into the same place). Problematic file: /usr/local/bin/doveadm *** Error code 70 (FreeBSD 11.1) -- Take my hand and I'll show you what was and will be.

2 2

PDFs getting mangled
by Adam Weinberger 24 Jan '18

24 Jan '18

Since upgrading to 2.3.0 / 0.5.0.1, incoming PDFs are getting mangled. It seems to be happening when I use vnd.dovecot.filter. When I comment out the block, things come through fine. My filter block looks like this: require "vnd.dovecot.filter"; filter "bogofilter_filter"; if header :contains "X-Bogosity" [ "Spam, tests=bogofilter, spamicity=1.00", "Spam, tests=bogofilter, spamicity=0.99" ] { fileinto "spam/totally"; stop; } elsif header :contains "X-Bogosity" "Spam," { fileinto "spam/probably"; stop; } elsif header :contains "X-Bogosity" "Unsure," { fileinto "spam/maybe"; stop; } filter/bogofilter_filter looks like this: #!/bin/sh cat /dev/stdin | /usr/local/bin/bogofilter -p -e -c /path/to/bogofilter.cf Images seem to come through fine. I didn't have this problem before the upgrade. It could easily be a problem with bogofilter, but I figured that I'd check here first to see if anybody else has an attachment-mangling issue with vnd.dovecot.filter. # Adam -- Adam Weinberger adamw(a)adamw.org http://www.adamw.org

3 7

Auth cache prevents login when non-password fields change
by Tom Sommer 23 Jan '18

23 Jan '18

The auth cache contains all fields returned by "password_query" (in itself a little odd), including the fields "nologin" and "reason". If a user is cached with "nologin=Y" and the database-source is changed so nologin condition is no longer true, then the cache prevents login for as long as the TTL remain? is there any way around this? Thanks -- Tom

1 0

rewrite copy to trash->delete->expunge mail with move??
by Chi Min Wang 23 Jan '18

23 Jan '18

Hello Everyone: When user got quota exceed and trying to delete mail. Most MUA's default "move to trash folder" will proceed by copy mail to trash folder,then by delete/expunge it(copy before expunge). Unfortunately this will not success due to quota exceed. Is there any way to let dovecot proceed it by IMAP MOVE instead of "copy to trash folder"->delete/expunge,Thanks!! *********************************************** CONFIDENTIALITY NOTICE:This e-mail and any attachments are confidential and may be legally privileged. If you are not the intended recipient, (i) please do not read or disclose to others, and (ii) please notify immediately the sender by reply mail, and (iii) please delete all copies of the email from your system. Failure to follow this process may be unlawful. We greatly appreciate your cooperation.

3 2

How to set autoexpunge rule for single user mailbox in dovecot conf?
by Alexey Murz Korepov 23 Jan '18

23 Jan '18

I need to set autoexpunge rule only for one single user and one mailbox (mail folder), so apply it only for user alice(a)example.com mailbox INBOX. How can I set namespace in dovecot config, that will apply only to single user? -- Best regards, Alexey Murz Korepov. Email, Jabber: murznn(a)gmail.com

3 2

LAYOUT=fs
by Jungersen, Danjel - Jungersen Grafisk ApS 23 Jan '18

23 Jan '18

2 5

Re: How to set autoexpunge rule for single user mailbox in dovecot conf?
by Aki Tuomi 23 Jan '18

23 Jan '18

You can do passdb { driver = passwd-file args = /path/to/some/file skip = unauthenticated } and put in this file alice@example.com::::::noauthenticate userdb_namespace/inbox/mailbox/INBOX/autoexpunge=31d or nopassword instead of noauthenticate. Please keep replies in-list. Aki On 23.01.2018 12:31, Alexey Murz Korepov wrote: > So, with using `passwd` driver as userdb - there are no ways to set > some dovecot single per-user setting? Change passwd to real user > database is too hard on working system. > Maybe exists some other ways to do auto expunge function for separate > mailbox folder on single user? > > As workaround, can I use command like `doveadm expunge` in cron job > for do this task? > --- > doveadm expunge -u alice(a)example.com <mailto:alice@example.com> > mailbox INBOX savedbefore 31d > --- > > On 23 January 2018 at 13:20, Aki Tuomi <aki.tuomi(a)dovecot.fi > <mailto:aki.tuomi@dovecot.fi>> wrote: > > Well, to have per-user settings, you need a userdb, that can > provide per-user settings. > > Aki > > > On 23.01.2018 12:19, Alexey Murz Korepov wrote: >> I can't understand what you mean in `return from userdb` - in >> which place of dovecot config files I can set this rule? I don't >> use separate userdb for dovecot users. >> Or maybe this must be set somewhere in user home folder? >> >> On 23 January 2018 at 13:17, Aki Tuomi <aki.tuomi(a)dovecot.fi >> <mailto:aki.tuomi@dovecot.fi>> wrote: >> >> No. >> >> The namespace name is irrelevant, it just has to be there. >> Assuming you have no other namespace config around, >> >> namespace inbox { >> inbox = yes >> mailbox INBOX { >> } >> } >> >> Then return from userdb, >> namespace/inbox/mailbox/INBOX/autoexpunge=31d >> >> The namespace config will be global (affects all users), but >> autoexpunge setting will be for alice only. >> >> Aki >> >> On 23.01.2018 12:08, Alexey Murz Korepov wrote: >>> Thanks, Aki, for quick answer! But can you give me please a >>> real example with user alice(a)example.com >>> <mailto:alice@example.com> and mailbox INBOX? In my setup I >>> have no user database, I use real linux users, managed via >>> Webmin/Virtualmin. >>> So I must set in `/etc/dovecot/conf.d/15-mailboxes.conf` >>> file something this? >>> ------ >>> namespace alice(a)example.com <mailto:alice@example.com> { >>> mailbox INBOX { >>> } >>> } >>> >>> namespace/alice(a)example.com/mailbox/INBOX/autoexpunge=31d >>> <http://alice@example.com/mailbox/INBOX/autoexpunge=31d> >>> >>> ------ >>> Or I must fill last string not in `mailboxes.conf` file, but >>> in some other place? >>> >>> On 23 January 2018 at 12:58, Aki Tuomi <aki.tuomi(a)dovecot.fi >>> <mailto:aki.tuomi@dovecot.fi>> wrote: >>> >>> Configure >>> >>> namespace <name> { >>> mailbox <name> { >>> } >>> } >>> >>> and return from userdb >>> >>> namespace/<name>/mailbox/<name>/autoexpunge=value >>> >>> Ak >>> >>> >>> On 23.01.2018 11:57, Alexey Murz Korepov wrote: >>>> I need to set autoexpunge rule only for one single user >>>> and one mailbox (mail folder), so apply it only for >>>> user alice(a)example.com <mailto:alice@example.com> >>>> mailbox INBOX. How can I set namespace in dovecot >>>> config, that will apply only to single user? >>>> >>>> -- >>>> Best regards, >>>> Alexey Murz Korepov. >>>> Email, Jabber: murznn(a)gmail.com <mailto:murznn@gmail.com> >>> >>> >>> >>> >>> -- >>> Best regards, >>> Alexey Murz Korepov. >>> Email, Jabber: murznn(a)gmail.com <mailto:murznn@gmail.com> >> >> >> >> >> -- >> Best regards, >> Alexey Murz Korepov. >> Email, Jabber: murznn(a)gmail.com <mailto:murznn@gmail.com> > > > > > -- > Best regards, > Alexey Murz Korepov. > Email, Jabber: murznn(a)gmail.com <mailto:murznn@gmail.com>

1 0

v2.3.0 release candidate released
by Timo Sirainen 23 Jan '18

23 Jan '18

https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz https://dovecot.org/releases/2.3/rc/dovecot-2.3.0.rc1.tar.gz.sig It's finally time for v2.3 release branch! There are several new and exciting features in it. I'm especially happy about the new logging and statistics code, which will allow us to generate statistics for just about everything. We didn't have time to implement everything we wanted for them yet, and there especially aren't all that many logging events yet that can be used for statistics. We'll implement those to v2.3.1, which might also mean that some of the APIs might still change in v2.3.1 if that's required. We also have new lib-smtp server code, which was used to implement SMTP submission server and do a partial rewrite for LMTP server. Please test these before v2.3.0 to make sure we don't have any bad bugs left! BTW. The v2.3.0 will most likely be signed with a new PGP key ED409DA1. Some of the larger changes: * Various setting changes, see https://wiki2.dovecot.org/Upgrading/2.3 * Logging rewrite started: Logging is now based on hierarchical events. This makes it possible to do various things, like: 1) giving consistent log prefixes, 2) enabling debug logging with finer granularity, 3) provide logs in more machine readable formats (e.g. json). Everything isn't finished yet, especially a lot of the old logging code still needs to be translated to the new way. * Statistics rewrite started: Stats are now based on (log) events. It's possible to gather statistics about any event that is logged. See http://wiki2.dovecot.org/Statistics for details * ssl_dh setting replaces the old generated ssl-parameters.dat * IMAP: When BINARY FETCH finds a broken mails, send [PARSE] error instead of [UNKNOWNCTE] * Linux: core dumping via PR_SET_DUMPABLE is no longer enabled by default due to potential security reasons (found by cPanel Security Team). + Added support for SMTP submission proxy server, which includes support for BURL and CHUNKING extension. + LMTP rewrite. Supports now CHUNKING extension and mixing of local/proxy recipients. + auth: Support libsodium to add support for ARGON2I and ARGON2ID password schemes. + auth: Support BLF-CRYPT password scheme in all platforms + auth: Added LUA scripting support for passdb/userdb. See https://wiki2.dovecot.org/AuthDatabase/Lua - Input streams are more reliable now when there are errors or when the maximum buffer size is reached. Previously in some situations this could have caused Dovecot to try to read already freed memory. - Output streams weren't previously handling failures when writing a trailer at the end of the stream. This mainly affected encrypt and zlib compress ostreams, which could have silently written truncated files if the last write happened to fail (which shouldn't normally have ever happened). - virtual plugin: Fixed panic when fetching mails from virtual mailboxes with IMAP BINARY extension. - Many other smaller fixes

13 26