- dovecot - dovecot.org

Segfault on selecting mailbox twice in a row
by John van der Kamp 15 Feb '18

15 Feb '18

Hi, I was doing some testing with new 2.2 changes. I’m using commit 15d5b8a from origin/master-2.2. The setup is using imapc to connect to a different imap server. If I use the following commands, dovecot will crash every time: A login username password B select inbox C select inbox The logfile snippit is at the end of the mail. Let me know if you need more information. John ---- Feb 15 15:43:06 imap(user1(a)domain.com): Debug: imapc: root=/tmp/tmpVuVTYr/dovecot1/users/user1(a)domain.com/imapc, index=, indexpvt=, control=, inbox=, alt= Feb 15 15:43:06 imap(user1(a)domain.com): Debug: maildir++: root=/tmp/tmpVuVTYr/dovecot1/users/user1(a)domain.com/imapc, index=, indexpvt=, control=, inbox=, alt= Feb 15 15:43:09 imap(user1(a)domain.com): Debug: INBOX: Mailbox opened because: SELECT Feb 15 15:43:12 imap(user1(a)domain.com): Debug: INBOX: Mailbox opened because: SELECT Feb 15 15:43:14 imap(user1(a)domain.com): Error: Log synchronization error at seq=2,offset=80372 for /tmp/tmpVuVTYr/dovecot1/users/user1(a)domain.com/imapc/.INBOX/dovecot.index: Append with UID 10, but next_uid = 10421 Feb 15 15:43:14 imap(user1(a)domain.com): Warning: fscking index file /tmp/tmpVuVTYr/dovecot1/users/user1(a)domain.com/imapc/.INBOX/dovecot.index Feb 15 15:43:14 imap(user1(a)domain.com): Warning: imapc(127.0.0.1:31513): Mailbox 'INBOX' state corrupted: Expunged message uid=10 reappeared - reconnecting (delay 0 ms) Feb 15 15:43:14 imap(user1(a)domain.com): Debug: imapc(127.0.0.1:31513): Disconnected Feb 15 15:43:14 imap(user1(a)domain.com): Debug: imapc(127.0.0.1:31513): Looking up IP address (reconnect_ok=false, last_connect=1518705794) Feb 15 15:43:14 imap(user1(a)domain.com): Debug: imapc(127.0.0.1:31513): Connecting to 127.0.0.1:31513 Feb 15 15:43:14 imap(user1(a)domain.com): Panic: file imapc-client.c: line 410 (imapc_client_mailbox_reconnect): assertion failed: (!box->reconnecting) Feb 15 15:43:14 imap(user1(a)domain.com): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x9fe12) [0x7f2a05301e12] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x9fefd) [0x7f2a05301efd] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f2a052924d1] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x99704) [0x7f2a0562f704] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(imapc_mailbox_set_corrupted+0xe3) [0x7f2a05628e63] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x97cf5) [0x7f2a0562dcf5] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x97d70) [0x7f2a0562dd70] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(imapc_mailbox_sync_init+0x8bd) [0x7f2a0562e89d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) [0x7f2a055e1c14] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7f2a055e1cb7] -> dovecot/imap(cmd_select_full+0x171) [0x413e51] -> dovecot/imap(command_exec+0x5c) [0x41b06c] -> dovecot/imap() [0x419662] -> dovecot/imap() [0x4196ec] -> dovecot/imap(client_handle_input+0x155) [0x419a85] -> dovecot/imap(client_input+0x82) [0x419ff2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f2a053180d2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f2a053197c9] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f2a0531816c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f2a05318318] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f2a0529cd93] -> dovecot/imap(main+0x302) [0x40cb92] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f2a04eb8830] -> dovecot/imap(_start+0x29) [0x40cd19] Feb 15 15:43:14 imap(user1(a)domain.com): Fatal: master: service(imap): child 9496 killed with signal 6 (core dumped)

2 1

Re: Problem with CentOS package for 2.3.0 and old dependency in systemd with clean install
by TG Servers 15 Feb '18

15 Feb '18

Ok thanks, for the moment : can I just delete the lines in the dovecot-init.service file or what is proposed here? It's no problem for me to make the necessary changes I just need to know which of them because I don't know if you just need this fiel internally or not. Thanks Am 14.02.2018 um 21:28 schrieb Aki Tuomi: > We are planning to fix this for 2.3.1 release. > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: TG Servers <srvrs(a)prvtmail.net> > Date: 14/02/2018 22:10 (GMT+02:00) > To: dovecot(a)dovecot.org > Subject: Re: Problem with CentOS package for 2.3.0 and old dependency in > systemd with clean install > > Hi, > > are there any news on this? > Or do we have to go the way install old dovecot/remove it or upgrade > instead of clean install? > Because as the ssl-params executaböe is missing in 2.3.0 I don't know > how else I should create it. If this file isn't even needed for 2.3.0 > can it be a file with any content eg 'touch > /var/lib/dovecot/ssl-parameters.dat' so that the file is just there and > dovecot-init.servive doesn't want to call the > /usr/libexec/dovecot/ssl-params executable? > > Thanks, > Thomas > > Am 02.02.2018 um 09:26 schrieb Aki Tuomi: >> >>> On February 2, 2018 at 5:09 AM TG Servers wrote: >>> >>> >>> Hi, >>> >>> you definitely have a problem with the packages out of your own repo for >>> version 2.3.0 and CentOS. >>> And this is only if you do a clean install, meaning there was no lower >>> dovecot version ever running on the system. >>> >>> If you want to 'systemctl start dovecot' it breaks with a dependency >>> error which comes from dovecot-init.service. >>> >>> dovecot-init.service : >>> [Unit] >>> Description=One-time Dovecot init service >>> ConditionPathExists=|!/var/lib/dovecot/ssl-parameters.dat >>> ConditionPathExists=|!/etc/pki/dovecot/certs/dovecot.pem >>> >>> [Service] >>> Type=oneshot >>> RemainAfterExit=no >>> ExecStart=/bin/sh -c '\ >>> if [ ! -f /etc/pki/dovecot/certs/dovecot.pem ]; \ >>> then\ >>> SSLDIR=/etc/pki/dovecot/ >>> OPENSSLCONFIG=/etc/pki/dovecot/dovecot-openssl.cnf >>> /usr/libexec/dovecot/mkcert.sh /dev/null 2>&1;\ >>> fi;\ >>> if [ ! -f /var/lib/dovecot/ssl-parameters.dat ]; \ >>> then\ >>> /usr/libexec/dovecot/ssl-params >/dev/null 2>&1; \ >>> fi' >>> >>> It wants to call /usr/libexec/dovecot/ssl-params if >>> /var/lib/dovecot/ssl-parameters.dat (which is deprecated now as I >>> understood) is not existing. >>> The problem is in 2.3.0 /usr/libexec/dovecot/ssl-params is not existent >>> anymore. >>> >>> This error does not occur if you for instance install 2.2.x from the >>> base repo, start it once, and then update the version from your repo. >>> This is because the ssl-parameters.dat was created with the old version >>> then. >>> >>> But this should not be the expected behaviour I think. It should be >>> possible to do a fresh install of 2.3.0 on a fresh system. >>> >>> Can you please get back to me on that? >>> >>> Thanks, >>> Thomas >> >> >> Thank you for reporting this, we'll look into it. >> >> --- >> Aki Tuomi >> >> Dovecot oy >>

2 1

doveadm with wildcard users?
by ＠lbutlr 15 Feb '18

15 Feb '18

# 2.2.33.2 (d6601f4ec): If I do doveadm index -u user "*" for a local user, it works fine, but if I do doveadm -u "*(a)sqldomain.tld" "*" or doveadm -u "*(a)localaliasdomain.tld" "*" or doveadm -u "*(a)localdomain.tld" "*" or doveadm -u "user(a)localdomain.tld" "*" it fails, … -u "user(a)sqldomain.tld" Seems to work <https://wiki2.dovecot.org/Tools/Doveadm/Index> -u user/mask Run the command only for the given user. It's also possible to use '*' and '?' wildcards (e.g. -u *(a)example.org)

2 4

Problem with CentOS package for 2.3.0 and old dependency in systemd with clean install
by TG Servers 14 Feb '18

14 Feb '18

Hi, you definitely have a problem with the packages out of your own repo for version 2.3.0 and CentOS. And this is only if you do a clean install, meaning there was no lower dovecot version ever running on the system. If you want to 'systemctl start dovecot' it breaks with a dependency error which comes from dovecot-init.service. dovecot-init.service : [Unit] Description=One-time Dovecot init service ConditionPathExists=|!/var/lib/dovecot/ssl-parameters.dat ConditionPathExists=|!/etc/pki/dovecot/certs/dovecot.pem [Service] Type=oneshot RemainAfterExit=no ExecStart=/bin/sh -c '\ if [ ! -f /etc/pki/dovecot/certs/dovecot.pem ]; \ then\ SSLDIR=/etc/pki/dovecot/ OPENSSLCONFIG=/etc/pki/dovecot/dovecot-openssl.cnf /usr/libexec/dovecot/mkcert.sh /dev/null 2>&1;\ fi;\ if [ ! -f /var/lib/dovecot/ssl-parameters.dat ]; \ then\ /usr/libexec/dovecot/ssl-params >/dev/null 2>&1; \ fi' It wants to call /usr/libexec/dovecot/ssl-params if /var/lib/dovecot/ssl-parameters.dat (which is deprecated now as I understood) is not existing. The problem is in 2.3.0 /usr/libexec/dovecot/ssl-params is not existent anymore. This error does not occur if you for instance install 2.2.x from the base repo, start it once, and then update the version from your repo. This is because the ssl-parameters.dat was created with the old version then. But this should not be the expected behaviour I think. It should be possible to do a fresh install of 2.3.0 on a fresh system. Can you please get back to me on that? Thanks, Thomas

2 3

Documentation Bug
by Andrew Beck 13 Feb '18

13 Feb '18

In https://wiki2.dovecot.org/Tools/Doveadm/Sync#section_arguments the destination list 5 possible options for the destination but in the page on migration https://wiki2.dovecot.org/Migration/Dsync it seems to use a sixth undocumented "imapc:" option for destination e.g. doveadm -o mail_fsync=never backup -R -u user@domain imapc: "imapc:" is also referenced on https://wiki2.dovecot.org/Migration/Gmail Is https://wiki2.dovecot.org/Tools/Doveadm/Sync#section_arguments missing this option? Andrew

3 2

deny passdb match messages logged only with auth_verbose=yes
by Marco Giunta 13 Feb '18

13 Feb '18

Hi at all, using deny passwd to restrict IMAP/POP3 access (https://wiki.dovecot.org/Authentication/RestrictAccess) I get deny passdb match messages: Feb 13 16:09:33 server-02 dovecot: auth: passwd-file(USERNAME,10.10.10.46,<9hzaYRllbsCTehgu>): User found from deny passdb only with auth_verbose=yes, sets global or defined in passdb block. But if I set auth_verbose=yes, for every user not present in passwd-file, Dovecot logs: Feb 13 16:09:57 server-02 dovecot: auth: passwd-file(USERNAME,10.10.10.46,<9hzaYRllbsCTehgu>): unknown user I know that if the account does not exist in the first passdb (deny passdb), then the error occur, even if it exists in the other passdb. This is normal, but auth_verbose shouldn't be used only to " Log unsuccessful authentication attempts and the reasons why they failed." ?? Again, I'm not a programmer, but 'auth_request_log_info' function in 'https://github.com/dovecot/core/blob/release-2.2.33/src/auth/auth-request.c' seems to log events only when 'auth_verbose=yes'. Is there another way to get deny passdb match messages, without enable verbose log ? Thanks, Marco -- Marco Giunta - ITCS SysAdmin Via Bonomea, 265 34136 - Trieste, Italy Tel: +39-040-3787-503 Fax: +39-040-3787-244

1 0

Certificate of package repository expired
by Marc Weustink 13 Feb '18

13 Feb '18

Hi, I just happened to do an apt update and got a notice that repo.dovecot.org was disabled. According to firefox: repo.dovecot.org uses an invalid security certificate. The certificate expired on dinsdag 13 februari 2018 12:48. The current time is dinsdag 13 februari 2018 16:39. Thanks, Marc

2 2

dovecot: master: Panic: kevent() failed: Invalid argument
by Adrian Gonzalez 13 Feb '18

13 Feb '18

Hello I've been seeing these panics with dovecot 2.2.31 (and older versions) under FreeBSD about once every 2-3 months or so. I know it's been discussed here in the list, but I didn't see a definitive fix. If it's been fixed, please ignore this post (sorry!). I finally managed to observe the bug on a server that has the patch that prints out the kevent() timespec values and got: Nov 29 16:46:27 mail dovecot: master: Panic: kevent(events=74, ts=59.1000000000) failed: Invalid argument Looking through the code, the panic is triggered in io_loop_handler_run_internal(). The timeout values for the kevent() call are obtained by calling io_loop_get_wait_time(), which in turn calls timeout_get_wait_time() (both in ioloop.c). The timeout is computed by subtracting the value returned by gettimeofday() from timeout->next_run, and it looks like in very rare cases the result in tv_r->tv_usec is 1000000uS (1 second). So, it seems that if gettimeofday() returns exactly 0 microseconds, and timeout->next_run has exactly 1000000 microseconds, the kevent() call will fail (1 in 1e12 chance?). I'm not entirely sure where timeout->next_run is computed, but looking at timeout_update_next(), it adjusts the seconds only if the microseconds are greater than 1000000: (ioloop.c, line 235) if (timeout->next_run.tv_usec > 1000000) { timeout->next_run.tv_sec++; timeout->next_run.tv_usec -= 1000000; } It seems like the comparison here should be greater or equal, but I'm not familiar enough with the code to say with absolute certainty. I added more debug error messages to the code in timeout_update_next(), and although I do see next_run->tv_usec at 1000000 several times a day, I haven't seen a panic since early December on any of our servers, so I decided to post this to see if this analysis makes sense.

2 1

Setting up sieve with spam learning
by ＠lbutlr 13 Feb '18

13 Feb '18

Cribbing from the dovecot site I recently added the following lines to my dovecot.conf: managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve plugin { imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocol imap { mail_plugins = " virtual imap_sieve" } Now, should I see the report-spam or report-ham sieve's logged anywhere? Because I am not seeing them.

2 4

"search return (min)" is very inefficient
by Jesus Cea 13 Feb '18

13 Feb '18

RFC 4731: " IMAP4 Extension to SEARCH Command for Controlling What Kind of Information Is Returned" https://tools.ietf.org/html/rfc4731 I do this (telnet session): """ a select proveedores/TUHS * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 8330 EXISTS * 0 RECENT * OK [UNSEEN 6085] First unseen. * OK [UIDVALIDITY 1395814416] UIDs valid * OK [UIDNEXT 8358] Predicted next UID * OK [HIGHESTMODSEQ 3149] Highest a OK [READ-WRITE] Select completed (0.264 + 0.000 + 0.263 secs). a search return (min) since 01-Feb-2018 * ESEARCH (TAG "a") MIN 8156 a OK Search completed (59.451 + 0.000 + 59.450 secs). """ Note that it needs 59 seconds to locate the first message matching the "since" criteria. This is really slow and unusable in practice. Things are worse with bigger mailboxes, of course. Am I missing some performance trick?. I am trying to locate the first message received after a particular date. I don't use the "date" header because it can be falsified and because I care about the receiving date and not the actual sending date. I guess dovecot is scanning every message in the folder (mailbox). Wouldn't those messages be stored in ascending "received" order?. Could that be used for bisection search to locate the right message?. Could that break if I copy/move/edit messages?. Is that the reason for such a awful performance?. Suggestions?. Thanks for your time and knowledge!. -- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz

1 0