- dovecot - dovecot.org

Re: BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish)
by J. Nick Koston 20 May '16

20 May '16

Hi Timo, The below solved the problem for me as far as I can tell I was getting a EPIPE about 1:30 dsync runs over the socket. I have received 0 in 10000 runs with this change. https://github.com/dovecot/core/pull/10 <https://github.com/dovecot/core/pull/10> You know the code much better then I do so you might have a better way? Also I have a commit to address the constant warnings when the prefix is set to INBOX. with the quota_quota plugin here: https://github.com/dovecot/core/pull/7 <https://github.com/dovecot/core/pull/7> Thanks -Nick Timo Sirainen tss at iki.fi <mailto:dovecot%40dovecot.org?Subject=Re%3A%20BUG%3F%3A%20Dsync%20over%20piped%20process%20%28DSYNC_RUN_TYPE_CMD%29%3A%20EPIPE%20if%20X%0A%09%28done%29%20is%20received%20in%20another%20write%20after%20F%20%28finish%29&In-Reply-To=%3CA5A2C866-9A65-439B-B232-231202B94525%40iki.fi%3E>Thu May 19 23:15:18 UTC 2016 > > On 19 May 2016, at 03:54, J. Nick Koston <nick at cpanel.net <http://dovecot.org/cgi-bin/mailman/listinfo/dovecot>> wrote: > > > > Hi, > > > > It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn’t received the X (done) response. > > If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. > > > > This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. > > Do you see this as causing some problem? I think this is intentional. The F(inish) tells the final response whether everything was OK or not, so all the work is done after that. The 'X' is described in code as: > > /* notify remote that we're closing. this is mainly to avoid > "read() failed: EOF" errors on failing dsyncs */ > o_stream_nsend_str(ibc->output, > t_strdup_printf("%c\n", items[ITEM_DONE].chr)); > > and on the receiving side: > > if (line[0] == items[ITEM_DONE].chr) { > /* remote cleanly closed the connection, possibly because of > some failure (which it should have logged). we don't want to > log any stream errors anyway after this. */ > dsync_ibc_stream_stop(ibc); > return DSYNC_IBC_RECV_RET_TRYAGAIN; > } > > So I think it only matters in error handling.

1 0

BUG?: Dsync over piped process (DSYNC_RUN_TYPE_CMD): EPIPE if X (done) is received in another write after F (finish)
by J. Nick Koston 20 May '16

20 May '16

Hi, It appears that dsync closes the file handle once it gets the F (finish) response even if it hasn’t received the X (done) response. If the X (done) response gets delivered in another write this will generate EPIPE in the connected process. This is much easier to replicate over SSL, however the trace is provided without SSL since its easier to see whats going on. Thanks J. Nick Koston # dovecot --version 2.2.24 (a82c823) dovecot-2.2.24-1.x86_64 F\n is in packet 1 X\n is in packet 2 STRACE of an the unexpected EPIPE: [pid 34753] read(4, "F\n", 524288) = 2 [pid 34753] write(1, "F\n", 2) = 2 [pid 34752] <... epoll_wait resumed> {{EPOLLIN, {u32=42323904, u64=42323904}}}, 3, 3350) = 1 [pid 34753] select(8, [0 4], NULL, NULL, {21600, 0} <unfinished ...> [pid 34752] read(10, "F\n", 1361) = 2 [pid 34752] write(9, "X\n", 2 <unfinished ...> [pid 34753] <... select resumed> ) = 1 (in [0], left {21599, 999925}) [pid 34752] <... write resumed> ) = 2 [pid 34753] read(0, <unfinished ...> [pid 34752] epoll_ctl(6, EPOLL_CTL_DEL, 10, {0, {u32=42323904, u64=42323904}} <unfinished ...> [pid 34753] <... read resumed> "X\n", 524288) = 2 [pid 34752] <... epoll_ctl resumed> ) = 0 [pid 34753] write(4, "X\n", 2 <unfinished ...> [pid 34752] close(9) = 0 [pid 34753] <... write resumed> ) = 2 [pid 34752] close(10 <unfinished ...> [pid 34753] select(8, [0 4], NULL, NULL, {21600, 0} <unfinished ...> [pid 34752] <... close resumed> ) = 0 [pid 34753] <... select resumed> ) = 2 (in [0 4], left {21599, 999998}) [pid 34752] epoll_wait(6, <unfinished ...> [pid 34753] read(4, "X\n", 524288) = 2 [pid 34753] write(1, "X\n", 2) = -1 EPIPE (Broken pipe) STRACE where its in one packet: F\nX\n is in a single packet [pid 34833] select(8, [0 4], NULL, NULL, {21600, 0}) = 1 (in [4], left {21599, 999966}) [pid 34833] read(4, "F\nX\n", 524288) = 4 [pid 34833] write(1, "F\nX\n", 4) = 4 [pid 34833] select(8, [0 4], NULL, NULL, {21600, 0} <unfinished ...> [pid 34832] <... epoll_wait resumed> {{EPOLLIN, {u32=38678464, u64=38678464}}}, 3, 3889) = 1 [pid 34833] <... select resumed> ) = 1 (in [4], left {21599, 999572}) [pid 34832] read(10, "F\nX\n", 1361) = 4

2 1

mailbox.auto ignores dovecot-uidlist.lock
by Tom Sommer 19 May '16

19 May '16

I'm trying to lock down a maildir from modifications using dovecot-uidlist.lock, but when a user with mailbox.auto = create logs in, then the folder is created regardless of dovecot-uidlist.lock existing or not. Is there no way to prevent dovecot from touching anything in a maildir folder? -- Tom

3 5

passwd mismatch
by Mike Ponto 19 May '16

19 May '16

I have been trying to follow the "instructions" at "http://blog.iandreev.com/?p=1975#dovecot" and they seem to be pretty good. everything works until I get to the part were I test the dovecot installation and it keeps failing. I get an AUTH fail error at the "telnet localhost 110" login commands I am trying to setup 3 domains: domain1.us domain2.com domain3.com I have 4 users entered in the /etc/dovecot/users file I followed the instructions for creating an encrypted password but don't think it is correct and I don't have the knowledge to say why. Just doesn't feel right. I admit I am over my head and the learning curve just got exponential - any help at all would be really appreciated. System: QEMU/KVM vitrual install Centos 7.01 basic install 4096 G Ram 4 proc 120 G HD unit is behind an IPCop firewall with port forwarding. internal IP 192.168.1.155 external IP on request have username in /etc/dovecot/users as such: mynamehere(a)mydomainhere.us: Turned on debuggin and log file shows: May 18 16:45:02 auth: Debug: auth client connected (pid=5639) May 18 16:45:16 auth: Debug: client in: AUTH 1 PLAIN service=pop3 secured session=2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB lip=::1 rip=::1 lport=110 rport=33742 $ May 18 16:45:16 auth: Debug: passwd-file(mynamehere@mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): lookup: user=mynamehere(a)mydomainhere.us file=/etc/dovecot/users May 18 16:45:16 auth: Info: passwd-file(mynamehere@mydomainhere.us,::1,<2pP4xSQzzgAAAAAAAAAAAAAAAAAAAAAB>): Password mismatch May 18 16:45:18 auth: Debug: client passdb out: FAIL 1 user=mynameherel(a)mydomainhere.us May 18 16:45:20 pop3-login: Info: Aborted login (auth failed, 1 attempts in 4 secs): user=<mynamehere(a)mydomain.us>, method=PLAIN, rip=::1, lip=::1, secured, session=<2pP4xSQzzgAAAAAA$

3 4

synchronous replication based on priority
by Sylvain Allemand 18 May '16

18 May '16

Hi, i see in this page (http://blog.dovecot.org/2012/02/dovecot-clustering-with-dsync-based.html) : "Optionally the replication plugin can also support synchronous replication of new mail deliveries. In this way it connects to replication-notify UNIX socket, tells it to replicate the user with sync (=highest) priority and waits until it is done or replication_sync_timeout occurs. The IMAP/LMTP client won't see an "OK" reply until the mail is replicated (or the replication has failed). The synchronous replication probably adds a noticeable delay, so it might not be acceptable for IMAP, but might be for LMTP." is this fonctionnality effective for LMTP ? If so, where is the config ? The wiki says that replication is asynchronous. Thanks for help, Best regards -- Sylvain Allemand <mailto:sylvain.allemand@math.u-bordeaux1.fr> Cellule Informatique Institut de Mathématiques de Bordeaux - UMR5251 05 40 00 68 86 www.math.u-bordeaux.fr/imb/cellule <http://www.math.u-bordeaux.fr/imb/cellule>

1 0

doveconf filter broken?
by Tom 18 May '16

18 May '16

I'm running dovecot 2.0.9 According to the man page, the command: # doveconf -f service=imap is supposed to show just the settings for imap. Is this correct? When I issue the command it shows everything. Why?

2 1

indexer-worker out of memory with vsz_limit=750 MB?!
by Peer Heinlein 18 May '16

18 May '16

We're running 2.2.24 with fts_lucene and I wonder why indexer-working can be out-of-memory even with vsz_limit=750 MB ?! Peer May 15 12:14:51 dobby4 dovecot: indexer-worker(user(a)example.com): Fatal: pool_system_realloc(134217728): Out of memory May 15 12:14:51 dobby4 dovecot: indexer-worker(user(a)example.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xaa4b7) [0x7f37027b84b7] -> /usr/lib/dovecot/libdovecot.so.0(+0xab851) [0x7f37027b9851] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f37027b89a6] -> /usr/lib/dovecot/libdovecot.so.0(+0xd0405) [0x7f37027de405] -> /usr/lib/dovecot/libdovecot.so.0(+0xa56e2) [0x7f37027b36e2] -> /usr/lib/dovecot/libdovecot.so.0(+0xa5813) [0x7f37027b3813] -> /usr/lib/dovecot/libdovecot.so.0(+0xa59e3) [0x7f37027b39e3] -> /usr/lib/dovecot/libdovecot.so.0(buffer_write+0x37) [0x7f37027b3cf5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10b7a6) [0x7f3702b617a6] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10c0c3) [0x7f3702b620c3] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_ext_intro+0x34d) [0x7f3702b625e5] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10fb49) [0x7f3702b65b49] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0x38) [0x7f3702b66262] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x5ba) [0x7f3702b669b2] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0xbf) [0x7f3702b59eb7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_refresh+0x1d) [0x7f3702b79e97] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x11cc33) [0x7f3702b72c33] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0x127) [0x7f3702b72dba] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit+0x1f) [0x7f3702b72c91] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_index_set_header+0x68) [0x7f37014ebfd0] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_index_set_last_uid+0x3b) [0x7f37014ec00d] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x4093) [0x7f3700686093] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_get_last_uid+0x81) [0x7f37014eb3e5] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11b66) [0x7f37014f3b66] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x11c40) [0x7f37014f3c40] -> /usr/lib/dovecot/modules/lib10_quota_plugin.so(+0xea0d) [0x7f3701936a0d] -> /usr/lib/dovecot/modules/lib01_acl_plugin.so(+0x11f19) [0x7f3701b50f19] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x75) [0x7f3702aa2db4] May 15 12:14:51 dobby4 dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for user(a)example.com May 15 12:14:51 dobby4 dovecot: indexer-worker(user(a)example.com): Fatal: master: service(indexer-worker): child 179030 returned error 83 (Out of memory (service indexer-worker { vsz_limit=750 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 2

Reloading dovecot and postfix after updates
by ＠lbutlr 17 May '16

17 May '16

Recently someone used webmin to update dovecot, which went fine. However, after the update no one could login to the mail server because of the mismatch in the versions between what was installed and what postfix was expecting. Other that bapping the person on the head and saying “don’t do that” is there anyway to force a reload of postfix after dovecot reloads? Like adding a postfix reload to the dovecot rc.d script or someway of linking them so that if either version changes they both get reloaded? (I don’t even use webmin myself, so maybe it’s even possible through it?) I mean, it’s unlikely to happen again at this point, but still… -- "...and Digby considered how much he liked salt..."

2 1

Errors on Cross-Protocol Locking nbmand (ZFS Solaris)
by Andrew Watkins 17 May '16

17 May '16

Hello, I get the following errors when I move large e-mails (i.e. about 15MB) into mail files. May 17 13:26:48 mailer3 dovecot: [ID 583609 mail.error] imap(andrew): Error: fdatasync() failed with mbox file /home/andrew/Mail/nbmandON/mbox: Permission denied May 17 13:26:48 mailer3 dovecot: [ID 583609 mail.error] imap(andrew): Error: ftruncate() failed with mbox file /home/andrew/Mail/nbmandON/mbox: Permission denied May 17 13:26:48 mailer3 dovecot: [ID 583609 mail.error] imap(andrew): Error: close() failed with mbox file /home/andrew/Mail/nbmandON/mbox: Permission denied Background I am using Solaris 11 ZFS & SMB for my users home files with Solaris SMB protocol mandatory locking (nbmand=on) If I turn nbmand=off then I do not get any errors, so it is to do with SMB locking. I know this may not be a dovecot error, but any pointers why it is doing this so that I can report it to Oracle. I tried creating a small C program to mimic the same problem but no luck, since I know it would make it easier to get Oracle to fix it. # /usr/local/dovecot/sbin/dovecot --version 2.2.24 (a82c823) Thanks for now, Andrew -- Andrew Watkins * Birkbeck, University of London * Computer Science * * http://notallmicrosoft.blogspot.com * * UKOUG Systems SIG Chair * * UKOUG Tech committee * * tel: 020 7631 6720 *

2 2

Unknown setting: mail_plugins
by Graf 17 May '16

17 May '16

Hello! At the start dovecot complains all occurrences |mail_plugins settings |in all configs. For example 20-lmtp.conf contents: |protocol lmtp { mail_plugins = $mail_plugins quota } # doveconf | grep '$mail_plugin_dir\|mail_plugins$' doveconf: Fatal: Error in configuration file /etc/dovecot/conf.d/20-lmtp.conf line 2: Unknown setting: mail_plugins || ||| If I insert directly those strings in dovecot.conf and comment the !include conf.d/*.conf , I have no errors: # doveconf | grep '$mail_plugin_dir\|mail_plugins$' mail_plugin_dir = /usr/lib64/dovecot mail_plugins = mail_plugins = " quota" My settings: |# dovecot --version 2.2.24 (a82c823) # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql sqlite Passdb: checkpassword ldap passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql| |# ls -l /usr/lib64/dovecot/ | grep quota -rwxr-xr-x 1 root root 990 may 12 18:52 lib10_quota_plugin.la -rwxr-xr-x 1 root root 78696 may 12 18:52 lib10_quota_plugin.so -rwxr-xr-x 1 root root 1061 may 12 18:52 lib11_imap_quota_plugin.la -rwxr-xr-x 1 root root 11584 may 12 18:52 lib11_imap_quota_plugin.so -rwxr-xr-x 1 root root 1026 may 12 18:52 lib20_quota_clone_plugin.la -rwxr-xr-x 1 root root 8992 may 12 18:52 lib20_quota_clone_plugin.so| Thank you in advance.

1 1