- dovecot - dovecot.org

indexer-worker panics with latest mercurial
by Laeeth Isharc 20 Mar '15

20 Mar '15

Hi. I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial. Thanks. Laeeth Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques ts for laeeth(a)laeeth.com Mar 12 20:48:39 indexer-worker(laeeth(a)laeeth.com): Fatal: master: service(indexe r-worker): child 24003 killed with signal 6 (core dumps disabled) Mar 12 20:49:01 dsync-local(laeeth(a)laeeth.com): Error: Couldn't lock /home/mail/ laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds Mar 12 20:49:16 indexer-worker(laeeth(a)laeeth.com): Error: fts_tika: PUT http://l ocalhost:9997/tika/ failed: 500 Server Error Mar 12 20:49:17 indexer-worker(rosie(a)kaleidicassociates.com): Warning: I/O leak: 0x7fc47d60fcf0 (line 127, fd 25) Mar 12 20:49:17 indexer-worker(rosie(a)kaleidicassociates.com): Panic: file ioloop .c: line 39 (io_add_file): assertion failed: (callback != NULL) Mar 12 20:49:17 indexer-worker(rosie(a)kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7fc47d63fc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7fc47d63fd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc47d5f02e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87bd0) [0x7fc47d650bd0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7fc47d65183e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7fc47d63db14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7fc47d614448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7fc47d60c876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7fc47c733159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7fc47c72f637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7fc47cf5dd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x614) [0x7fc47cf5e3c4] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7fc47cf6314e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7fc47d8ef429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fc47d65155c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7fc47d65249b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fc47d6515c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fc47d651648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fc47d5f5703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 15 reque sts for rosie(a)kaleidicassociates.com Mar 12 20:49:17 indexer-worker(rosie(a)kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24022 killed with signal 6 (core dumps disabled) Mar 12 20:49:17 indexer-worker(laeeth(a)kaleidicassociates.com): Warning: I/O leak: 0x7f22c4d4dcf0 (line 127, fd 23) Mar 12 20:49:17 indexer-worker(laeeth(a)kaleidicassociates.com): Panic: file ioloop.c: line 829 (io_loop_move_io): assertion failed: ((old_io->condition & IO_NOTIFY) == 0) Mar 12 20:49:17 indexer-worker(laeeth(a)kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f22c4d7dc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f22c4d7dd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f22c4d2e2e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x888ba) [0x7f22c4d8f8ba] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f22c4d7bb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f22c4d52448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7f22c4d4a876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7f22c3e71159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7f22c3e6d637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7f22c469bd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x123) [0x7f22c469bed3] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7f22c46a114e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7f22c502d429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io _loop_call_io+0x4c) [0x7f22c4d8f55c] -> /usr/lib64/dovecot/libdovecot.s o.0(io_loop_handler_run_internal+0xbb) [0x7f22c4d9049b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f22c4d8f5c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f22c4d8f648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f22c4d33703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 6 requests for laeeth(a)kaleidicassociates.com Mar 12 20:49:17 indexer-worker(laeeth(a)kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24020 killed with signal 6 (core dumps disabled) Mar 12 20:49:26 indexer-worker(laeeth(a)laeeth.com): Warning: I/O leak: 0x7f0d5aa00cf0 (line 127, fd 23) Mar 12 20:49:26 indexer-worker(laeeth(a)laeeth.com): Panic: file ioloop-iolist.c: line 22: unreached Mar 12 20:49:26 indexer-worker(laeeth(a)laeeth.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f0d5aa30c40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f0d5aa30d1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0d5a9e12e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x88a0d) [0x7f0d5aa42a0d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f0d5aa4319b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87b61) [0x7f0d5aa41b61] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f0d5aa4283e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f0d5aa2eb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f0d5aa0

2 1

[Dovecot] imapc - different settings for different namespaces
by Timo Sirainen 20 Mar '15

20 Mar '15

Originally I was planning on allowing all kinds of mail settings inside namespace {}, including imapc_* settings. But that's a bit difficult to implement (although I think it will happen some day). So for now I was thinking: imapc foo { host = imap.foo.com master_user = foomaster password = foopass } imapc bar { host = imap.bar.com ... } namespace { prefix = foo/ location = imapc:foo } namespace { prefix = bar/ location = imapc:bar } So basically if the path after imapc isn't absolute (/path or ~/path), then treat the path as the imapc section name and use its settings. Anyone have any improvement suggestions? I was planning on adding this to v2.2.3. I'm not sure if the section names should be "imapc" or something else (e.g. imap_remote {}).

2 2

core not dumped
by James 20 Mar '15

20 Mar '15

There is some underlying problem with imap-login but in order to debug it I'd like to see a core file. On error dovecot reports "core not dumped". If I put "abort()" in the code at the point it is reporting then a core file is written, so there is no OS reason why a core can not be written. It appears to be dovecot's choice to control the child. In an attempt to obtain a core file I have: service imap-login { chroot = drop_priv_before_exec = yes user = dovecot } How can I make imap-login to produce a core file? Syslog error report: Mar 18 09:22:28 MAILHOST dovecot: [ID 583609 mail.crit] imap-login: Fatal: master: service(imap-login): child 10812 killed with signal 11 (core not dumped) [last ip=111.222.333.444] Should anyone care at this point (before I've looked at a core file) the error is related to; imap-login: Disconnected (no auth attempts in 0 secs): user=<>, James.

3 4

Re: fast doveadm search results
by Joseph Tam 19 Mar '15

19 Mar '15

Vu Ngoc VU <vvu(a)mcra.fr> writes: > My goal is to be able to find mails quickly with "doveadm search". > We are using dovecot in a corporate. And some managers, sometimes ask to delete some messages. > These messages are "unwanted" ones: mistakenly sent, defamatory... > > So, that situations appear rarely, but when asked, they expected me to > deleted quickly these messages from few to thousands of users > mailboxes. > > Is that possible to change behaviour of Dovecot, some headers like > sender, message-id, date are stored in cache for at least 48h? (7 days > would be better). So is this a fair summary of what you are asking: you want to load dovecot's caches with searchable items (i.e. specified headers) *and* have them expire and removed from caches after a preset amount of time? Caches are persistent: once an item is cached, it stays in the cache. I can't think of a good way to purge them without a lot of bother, but maybe you don't really need this. If you want to preload the cache with searchable items, you can periodically run a search/fetch command, which will load the cache with headers you want. http://www.dovecot.org/list/dovecot/2010-October/053521.html Maybe you can even add these items into the cache upon delivery, but someone more knowlegable than I would have to help you with that. The FTS/Solr plugin might also help. You seem to be optimizing the wrong problem. Reading between the lines of your message, your management wants a technical solution to a human problem (someone did an oops). Joseph Tam <jtam.home(a)gmail.com>

1 0

Released Pigeonhole v0.4.7 for Dovecot v2.2.16
by Stephan Bosch 19 Mar '15

19 Mar '15

Hello Dovecot users, Here is the final 0.4.7 release. No changes were committed since the last release candidate. Changelog v0.4.7: * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output about binary up-to-date checking. + Added script metadata to binary dump output. - Fixed Sieve script binary up-to-date checking by normalizing the script location. - The Sieve interpreter now flushes the duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. - file script storage: Restructured storage initialization to address backwards compatibility issues. - dict script storage: Fixed small memory allocation bug. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.tar… http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.tar… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

check if anvil is working
by Leonardo Rodrigues 19 Mar '15

19 Mar '15

Hi, Is there any way of making sure the anvil service, used to penalty login fails, is working ? I dont see anything on the logs regarding it neither have it configured. It's not on my configuration files, but it can be seen on a 'dovecot -a' dump. Should it be logging something ? Should i have enabled it somehow ? Thanks for the answers. [root@correio log]# dovecot --version 2.2.13 (from dovecot -a dump) service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } -- Atenciosamente / Sincerily, Leonardo Rodrigues Solutti Tecnologia http://www.solutti.com.br Minha armadilha de SPAM, NÃO mandem email gertrudes(a)solutti.com.br My SPAMTRAP, do not email it

1 0

fast doveadm search results
by Vu Ngoc VU 19 Mar '15

19 Mar '15

hello, I've posted same question here months ago. But obtaine no answer, even not a "you question is stupid". So I don't know what to think about that. My goal is to be able to find mails quickly with "doveadm search". We are using dovecot in a corporate. And some managers, sometimes ask to delete some messages. These messages are "unwanted" ones: mistakenly sent, defamatory... So, that situations appear rarely, but when asked, they expected me to deleted quickly these messages from few to thousands of users mailboxes. Is that possible to change behaviour of Dovecot, some headers like sender, message-id, date are stored in cache for at least 48h? (7 days would be better). I've read this page : http://wiki2.dovecot.org/Design/Indexes/Cache but cannot understand every parts. "A specific decision can be forced by ORing it with MAIL_CACHE_DECISION_FORCED." Is OR in this sentence like AND/OR comparators? Where to put "MAIL_CACHE_DECISION_FORCED"? "mail-cache-decisions.c file contains the rules how Dovecot changes the decisions." <-- is it in this file? My setup is 2.2.9 on Ubuntu 14.04 running on OpenVZ containers with 2 Director and 2 imap/pop3 backends. The storage is shared via NFS. Is this the performance problem? If dovecot has metadata I want in its cache, I cannot see how this is a problem. Can these settings be useful for my case? mail_always_cache_fields = mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_never_cache_fields = imap.envelope Thanks.

2 2

Support for multiple passwords?
by Conrad Kostecki 18 Mar '15

18 Mar '15

Hi! Currently, the passwords are stored in plaintext for my dovecot, as I am still using cram-md5 AND digest-md5. I have still to offer that, as I have some deprecated clients, therefore, I am unable to hash at least those passwords for that accounts. I've found on the Wiki: > In future it's possible that Dovecot could support multiple passwords > in different schemes for a single user. Is there any news about this? Are there still any plans to support this maybe in future? For my understanding, that would solve my problem, that I could define a password in both schemes (cram and digest) and don't have to use plaintext password? Cheers Conrad

3 6

passwd file for quota
by Cliff Hayes 18 Mar '15

18 Mar '15

I need to implement quota. I read as much as I could including http://wiki2.dovecot.org/HowTo/SimpleVirtualInstall But my current /etc/passwd file does not have the password in it as shown in dovecot docs ... there is an x where the password should be. I created a program that creates a passwd file in the proper format with the proper extra quota fields for all users (except it has an x where the password should be). I plan to store it in /etc/dovecot How do I get dovecot to stop looking at /etc/passwd and start looking at /etc/dovecot/passwd? And hopefully that will not affect how it gets the password (pam) because when users change their password from webmail of course dovecot will need to be looking in the same place. # dovecot -n output # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 Scientific Linux release 6.5 (Carbon) auth_failure_delay = 5 secs auth_username_format = %Ln auth_verbose_passwords = plain first_valid_uid = 9 mail_access_groups = mail mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_after = /dovecotSieveAfter/ sieve_dir = ~/sieve } protocols = imap lmtp sieve service managesieve-login { inet_listener sieve { port = 4190 } } ssl_ca = </etc/pki/dovecot/certs/intermediate20150202.crt ssl_cert = </etc/pki/dovecot/certs/avalon20150202.crt ssl_key = </etc/pki/dovecot/private/avalon20150129.key ssl_verify_client_cert = yes userdb { driver = passwd } protocol lmtp { log_path = /var/log/dovecot-lmtp.log mail_plugins = " sieve" } protocol lda { log_path = /var/log/dovecot-lda.log mail_plugins = " sieve" }

2 4

Roundcubemail problem with Dovecot
by Robert Moskowitz 18 Mar '15

18 Mar '15

I am getting "Connection to storage server failed." in roundcubemail. On doing a google search If find: http://help.directadmin.com/item.php?id=473 Which puts the problem at dovecot. I did get a dovecot update back on 3-5 and that would match to my one roundcube mail user complaining and the delays I am seeing with thunderbird and outlook users. So with my setup, I am trying to figure what are the index files to delete. Say for myself, I am seeing: /home/vmail/htt-consult.com/rgm/dovecot.index /home/vmail/htt-consult.com/rgm/dovecot.index.cache /home/vmail/htt-consult.com/rgm/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Drafts/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Sent/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Spam/dovecot.index.log /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index.cache /home/vmail/htt-consult.com/rgm/.Trash/dovecot.index.log Do I delete all of these? The index.cache files are quite large.

1 1