- dovecot - dovecot.org

Dovecot crash
by JCA 01 Dec '18

01 Dec '18

This happening when my Thunderbird client is trying to establish a connection with a Dovecot server. Some background first: 1) I am running Thunderbird 6.20.1 from a Linux client. 2) Other clients (e.g. Maildroid in my Android phone) do not have any issues. 3) The Dovecot software is version 2.2.9, running in a Linux server. 4) This may be the root of the problem: at some point, the Dovecot software was changed to version 2.3.1, and the reverted to 2.2.9. The crash started to appear when this reversion took place. My guess is that upgrading to 2.3.1 will eliminate this problem. However, in case that does not solve the problem, I would be grateful if anybody could throw some light into exactly what is going on here, and whether it can be fixed even without the need of upgrading to 2.3.1. What follows are the traces generated by Dovecot 2.2.9 when a connection is attempted from Thunderbird, with usernames and IP addresses disguised for privacy, and some blank lines inserted for readability: Nov 28 13:36:20 imap-login: Info: Login: user=<xyz>, method=PLAIN, rip=xxx.yyy.z zz.154, lip=aaa.bbb.ccc.7, mpid=29245, TLS, session=<q7jNhL97qQDMhbWa> Nov 28 13:36:20 imap(xyz): Panic: file mail-index-sync-keywords.c: line 227 (key words_update_records): assertion failed: (data_offset >= sizeof(struct mail_index_record)) Nov 28 13:36:20 imap(xyz): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so. 0(+0x65626) [0xb75ae626] -> /usr/lib/dovecot/libdovecot.so.0(+0x6569f) [0xb75ae6 9f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb756180e] -> /usr/lib/dove cot/libdovecot-storage.so.0(mail_index_sync_keywords+0x707) [0xb76cb6b7] -> /usr /lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0xf7) [0xb76cc167] - > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x2a4) [0xb76cd22 4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0x57c) [0xb76bcb9c ] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb7c2a) [0xb76b7c2a] -> /usr/lib /dovecot/libdovecot-storage.so.0(+0xb7daa) [0xb76b7daa] -> /usr/lib/dovecot/libd ovecot-storage.so.0(mail_index_open+0x114) [0xb76b7f54] -> /usr/lib/dovecot/libd ovecot-storage.so.0(index_storage_mailbox_open+0xab) [0xb76a808b] -> /usr/lib/do vecot/libdovecot-storage.so.0(+0x5a481) [0xb765a481] -> /usr/lib/dovecot/libdove cot-storage.so.0(+0x5a6f2) [0xb765a6f2] -> /usr/lib/dovecot/libdovecot-storage.s o.0(+0x7a017) [0xb767a017] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_o pen+0x26) [0xb767a1d6] -> dovecot/imap(client_open_save_dest_box+0x79) [0x805dca 9] -> dovecot/imap() [0x8052678] -> dovecot/imap(command_exec+0x32) [0x805d7a2] -> dovecot/imap() [0x805c746] -> dovecot/imap() [0x805c820] -> dovecot/imap(clie nt_handle_input+0x125) [0x805cad5] -> dovecot/imap(client_input+0x71) [0x805ce91 ] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0xb75c1292] -> /usr /lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xdb) [0xb75c22cb] -> /usr/lib/ dovecot/libdovecot.so.0(io_loop_run+0x48) [0xb75c0cd8] -> /usr/lib/dovecot/libdo vecot.so.0(master_service_run+0x2d) [0xb756728d] -> dovecot/imap(main+0x2bf) [0x 8066fef] -> /lib/libc.so.6(__libc_start_main+0xe6) [0xb73cfdb6] Nov 28 13:36:20 imap(xyz): Fatal: master: service(imap): child 29245 killed with signal 6 (core dumps disabled) Nov 28 13:36:20 imap-login: Info: Login: user=<xyz>, method=PLAIN, rip=xxx.yyy.z zz.154, lip=aaa.bbb.ccc.7, mpid=29247, TLS, session=<xPzShL97lADMhbWa> Nov 28 13:36:21 imap(xyz): Panic: file mail-index-sync-keywords.c: line 227 (key words_update_records): assertion failed: (data_offset >= sizeof(struct mail_inde x_record)) Nov 28 13:36:21 imap(xyz): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so. 0(+0x65626) [0xb7757626] -> /usr/lib/dovecot/libdovecot.so.0(+0x6569f) [0xb77576 9f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb770a80e] -> /usr/lib/dove cot/libdovecot-storage.so.0(mail_index_sync_keywords+0x707) [0xb78746b7] -> /usr /lib/dovecot/libdovecot-storage.so.0(mail_index_sync_record+0xf7) [0xb7875167] - > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x2a4) [0xb787622 4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_map+0x57c) [0xb7865b9c ] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xb7c2a) [0xb7860c2a] -> /usr/lib /dovecot/libdovecot-storage.so.0(+0xb7daa) [0xb7860daa] -> /usr/lib/dovecot/libd ovecot-storage.so.0(mail_index_open+0x114) [0xb7860f54] -> /usr/lib/dovecot/libd ovecot-storage.so.0(index_storage_mailbox_open+0xab) [0xb785108b] -> /usr/lib/do vecot/libdovecot-storage.so.0(+0x5a481) [0xb7803481] -> /usr/lib/dovecot/libdove cot-storage.so.0(+0x5a6f2) [0xb78036f2] -> /usr/lib/dovecot/libdovecot-storage.s o.0(+0x7a017) [0xb7823017] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_o pen+0x26) [0xb78231d6] -> dovecot/imap(client_open_save_dest_box+0x79) [0x805dca 9] -> dovecot/imap() [0x8052678] -> dovecot/imap(command_exec+0x32) [0x805d7a2] -> dovecot/imap() [0x805c746] -> dovecot/imap() [0x805c820] -> dovecot/imap(clie nt_handle_input+0x125) [0x805cad5] -> dovecot/imap(client_input+0x71) [0x805ce91 ] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x42) [0xb776a292] -> /usr /lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xdb) [0xb776b2cb] -> /usr/lib/ dovecot/libdovecot.so.0(io_loop_run+0x48) [0xb7769cd8] -> /usr/lib/dovecot/libdo vecot.so.0(master_service_run+0x2d) [0xb771028d] -> dovecot/imap(main+0x2bf) [0x 8066fef] -> /lib/libc.so.6(__libc_start_main+0xe6) [0xb7578db6] Nov 28 13:36:21 imap(xyz): Fatal: master: service(imap): child 29247 killed with signal 6 (core dumps disabled)

3 6

Errors wih fts-solr
by Riccardo Bicelli 30 Nov '18

30 Nov '18

Hello, I recently upgraded my solr installation to 6.6.5 When searching through dovecot solr throws this this error: solr Bad contentType for search handler :text/xml I'm running dovecot 2.2.10 on CentOS 7.5. Regards Riccardo

3 4

Replacing email envelope
by jon.sp2＠wp.pl 30 Nov '18

30 Nov '18

Hi, Could you please advice how I can replace envelope of a message received on my server? Best would be using sieve. Simple case - email A(a)example.com receives message, which I'd like to repack into new envelope (I want to have control what values in the envelope will be there - from, to, replay-to et.c) and send to B(a)test.com . At first I wanted to use editheader extension, but it is not allowing me to modify From or To values as far as I can tell. Another idea was to use envelope, but I don't see any way of editing values. Only reading. Now, I am thinking of writing script and from sieve it will be passed there, where I'll do message parsing and send new message using mutt or something similar. Frankly speaking I suppose there is much better solution to that and I'd appreciate any advice on the topic. Thanks, Jon

5 5

rspamd / imapsieve script is not executed/called
by Michael Ludwig 30 Nov '18

30 Nov '18

Hi to all of you, as I am struggling around with this a week or so I'm afraid I have to ask for help here. The problem is sitting right in front of the computer, as most of the time. ;-) So let me explain: Dovecot is working properly for IMAP with LE certificates, so I thought it's time to continue with Christoph Haas tutorial primarily and other blogs and tutorials for setting up the server. Next part would be to get rspamd learning what mails the user puts into junk folder. I followed the tutorial, but it will not work. Also I tried to get into debug mode, but I feel that dovecot is not really logging too much. Server is freshly set up Ubuntu 18.04.1, latest patchlevel with packages mostly from Ubuntu sources, except rspamd, but this was added as ppa if my remembering is correct. This is what I get on syslog, whilst *.* is being put at /var/log/allmessages. ######################################### Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: INBOX/Junk: Mailbox opened because: UID move Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: imapsieve: mailbox INBOX/Junk: MOVE event Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: sieve: Pigeonhole version 0.4.21 (92477967) initializing Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.4.21 (92477967) loaded Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.4.21 (92477967) loaded Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: imapsieve: Static mailbox rule [1]: mailbox=`Junk' from=`*' causes=(COPY) => before=`file:/etc/dovecot/sieve/report-spam.sieve' after=(none) Nov 25 23:30:50 myserver dovecot: imap(info(a)mydomain.tld): Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`Junk' causes=(COPY) => before=`file:/etc/dovecot/sieve/report-ham.sieve' after=(none) ######################################### So as far as I can interpret there is absolutely no script called when I move a mail from inbox to junk folder in Thunderbird. Here is the output of dovecot -n but I don't exactly if this is enough, so please give me an info if you need more output. ######################################### # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.15.0-39-generic x86_64 Ubuntu 18.04.1 LTS ext4 auth_mechanisms = plain login lmtp_rcpt_check_quota = yes lmtp_save_to_detail_mailbox = yes mail_debug = yes mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = " quota" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe autoexpunge = 90 days special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 30 days special_use = \Trash } prefix = INBOX/ separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * quota_grace = 20%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is over quota quota_status_success = DUNNO recipient_delimiter = + sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/sieve-after/ sieve_global = /etc/dovecot/sieve sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_debug = yes } protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 22466 } } ssl_cert = </etc/letsencrypt/live/myserver.mydomain.tld/fullchain.pem ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { args = uid=vmail gid=vmail home=/var/vmail/%d/%n driver = static } protocol lmtp { mail_plugins = " quota quota sieve" postmaster_address = postmaster(a)mydomain.tld } protocol imap { mail_plugins = " quota imap_sieve imap_quota" } ######################################### Would be really great if somebody knows what the he** I am doing wrong. Thank you for taking the time to go through this. Michael

2 3

Error logs with virtual folder after upgrading to 2.3.4
by Maria Arrea 30 Nov '18

30 Nov '18

4 3

Best way of merging mbox files
by Marc Roos 30 Nov '18

30 Nov '18

When concatenating mbox files like described here https://xaizek.github.io/2013-03-30/merge-mbox-mailboxes/. You will end up with an 'unsorted' mbox file. Is this going to be a problem esspecially when they are large >2GB's and new emails will be written to it? The email client nicely sorts the message from folder A "foldera 5 last" as last, but of course the mbox is not like this. Is there a better solution for merging files? Having: A folder A with messages: From Subject Received Size test foldera 1 16:18 665 B test foldera 2 16:18 665 B test foldera 3 16:18 665 B test foldera 4 16:18 665 B test foldera 5 last 16:29 670 B A folder B with messages: From Subject Received Size test folderb 1 16:23 665 B test folderb 2 16:24 665 B test folderb 3 16:24 665 B test folderb 4 16:24 665 B test folderb 5 16:24 665 B [@ mail] cat .foldera .folderb > .folderc Getting a folder C with messages: From Subject Received Size test foldera 1 16:18 665 B test foldera 2 16:18 665 B test foldera 3 16:18 665 B test foldera 4 16:18 665 B Mail System Internal Data DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA 16:19 454 B test folderb 1 16:23 665 B test folderb 2 16:24 665 B test folderb 3 16:24 665 B test folderb 4 16:24 665 B test folderb 5 16:24 665 B test foldera 5 last 16:29 670 B

3 3

"merging" folders Sent-Items Sent-Messages Sent
by Marc Roos 29 Nov '18

29 Nov '18

I have a running server and would like to clean up a bit, and merge sent folders and trash folders etc. Is this best done via the mailbox alias plugin like mentioned here? Or is there a better way? I prefer to have one folder listed at the client, and not "Unfortunately it also means that some clients will download the same mails to local cache multiple times." https://wiki2.dovecot.org/Plugins/MailboxAlias dovecot-2.2.10-8.el7.x86_64 CentOS Linux release 7.5.1804 (Core)

1 0

Solr: URI Too Long
by azurit＠pobox.sk 29 Nov '18

29 Nov '18

Hi, i'm quite often getting this error: Error: fts_solr: Lookup failed: 414 URI Too Long Most web servers are limiting URI length to 8192 characters. Why not to use POST instead of GET when communicating with Solr? Solr is able to handle this ok, also see: https://dzone.com/articles/solr-select-query-get-vs-post azur

1 0

determine why mail clients download mails 2nd time
by Poliman - Serwis 29 Nov '18

29 Nov '18

I have strange problem. In last Friday I had a problem with dovecot. It didn't see directories with mailboxes in /var/vmail. Somehow it crashed because probably two things (a lot of this type of lines in mail.err file): *Nov 23 07:05:02 s1 dovecot: lda(poli(a)poli.li <poli(a)poli.li>): Error: User initialization failed: Initializing mail storage from mail_location setting failed: mkdir(/var/vmail/poli.li/poli/Maildir <http://poli.li/poli/Maildir>) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +w perm: /var/vmail, dir owned by 0:0 mode=0755)* and maybe *Nov 12 09:27:59 s1 dovecot: imap(artur(a)firmastanaszek.pl <artur(a)firmastanaszek.pl>): Error: Corrupted index cache file /var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache <http://firmastanaszek.pl/artur/Maildir/dovecot.index.cache>: Broken MIME parts for mail UID 559 in mailbox INBOX: Cached MIME parts don't match message during parsing: Cached header size mismatch (parts=)Nov 12 09:27:59 s1 dovecot: imap(artur(a)firmastanaszek.pl <artur(a)firmastanaszek.pl>): Error: unlink(/var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache <http://firmastanaszek.pl/artur/Maildir/dovecot.index.cache>) failed: No such file or directory (in mail-cache.c:28)* and after this event probably all mailboxes download again all emails from 1.11 to 22.11. They were duplicated in mail clients. -- *Pozdrawiam / Best Regards* *Piotr Bracha*

2 3

Bug in logging in debug mode
by Xavier 29 Nov '18

29 Nov '18

Hello, This is dovecot 2.3.4 (0ecbaf23d) If you set "mail_debug = yes" in 10.logging.conf, there's a bug in logging, which breaks log parsers : > 2018-11-29 00Debug: :41:57 lda(xavier): Effective uid=501, gid=20, home=/home/xavier ^^^^ Regards, Xavier -- Xavier Humbert - sysadmin & network engineer

1 0