January 2015 - dovecot

HG e3640ccaa76d crashes upon IMAP SEARCH
by Thomas Leuxner 15 Jan '15

15 Jan '15

$ doveconf -n | head # 2.2.15 (e3640ccaa76d): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 # OS: Linux 3.16.0-0.bpo.4-amd64 x86_64 Debian 7.8 $ /usr/lib/dovecot/imap -u tlx(a)leuxner.net * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE NOTIFY QUOTA ACL RIGHTS=texk] Logged in as tlx(a)leuxner.net a select "Shops/Amazon" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk NotJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Seen \Draft $NotJunk NotJunk \*)] Flags permitted. * 491 EXISTS * 0 RECENT * OK [UNSEEN 491] First unseen. * OK [UIDVALIDITY 1213986801] UIDs valid * OK [UIDNEXT 568] Predicted next UID * OK [HIGHESTMODSEQ 1029] Highest a OK [READ-WRITE] Select completed (0.003 secs). b search charset utf-8 body "Sleeve" imap(tlx(a)leuxner.net): Panic: file message-decoder.c: line 346 (message_decode_body): assertion failed: (ctx->translation_size <= sizeof(ctx->translation_buf)) imap(tlx(a)leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7749f) [0x7ffe68e6d49f] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7ffe68e6d57a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ffe68e1df65] -> /usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x688) [0x7ffe68e5c858] -> /usr/lib/dovecot/libdovecot.so.0(message_search_more+0xa1) [0x7ffe68e60881] -> /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0x5b) [0x7ffe68e60a4b] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa9b7b) [0x7ffe69187b7b] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x45) [0x7ffe6915d675] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa951a) [0x7ffe6918751a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xab76a) [0x7ffe6918976a] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0xe2) [0x7ffe69189e32] -> /usr/lib/dovecot/modules/lib90_stats_plugin.so(+0x2daa) [0x7ffe66dc3daa] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) [0x7ffe691627a7] -> /usr/lib/dovecot/imap() [0x41da2f] -> /usr/lib/dovecot/imap() [0x41e0a2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xaa) [0x7ffe68e7f6ca] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x69) [0x7ffe68e804e9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7ffe68e7f549] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ffe68e7f5c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ffe68e23433] -> /usr/lib/dovecot/imap(main+0x2a1) [0x40c591] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7ffe68a88ead] -> /usr/lib/dovecot/imap() [0x40c6f5] I tried to make it dump a core but failed... Regards Thomas

2 2

attn rick van rein (midget)
by Laeeth Isharc 14 Jan '15

14 Jan '15

From 11 Oct 2013 (Rick van Rein) "Hello, I love Dovecot, but when developing a small IMAP tool, I ran into searching behaviour can easily be optimised. Please forgive a rather detailed suggestion. This was on Dovecot 1.2.15 on Debian Squeeze. My tool? It's called "midget" and retrieves documents from an IMAP box based on their mid: or cid: identifier, as per RFC 2392. I thought this would be useful to retrieve email attachments into a remote shell environment. When using Kerberos, the credentials and a strong hint of one's mail address are already present anyway, DNS SRV does the rest. If you want to see the early code, I'll be happy to share it here." Hi Rick. Thanks for the suggested optimisation - I am not sure if it is you who I have to think for contributing towards making IMAP search fast again. In any case, if you see this I would be interested to see the source for midget if you are willing to share. If you don't want to post to the list then my email is laeeth at laeeth.com Thanks. Laeeth

1 0

dovecot.org slow & imap search now blazing fast
by Laeeth Isharc 14 Jan '15

14 Jan '15

Hi. In case you were not aware, I wanted to let you know that dovecot.org has been very slow (often almost unusable) for at least several days now. Also, I upgraded my dovecot from 2.2.10 to 2.2.15. Thank you for fixing IMAP search. I have c. 2mm messages, and it had become unusable before but it is now very fast. (I saw a reference to an optimisation on the mailing list from back in Oct 2013, so maybe that was it). Regards, Laeeth

1 0

pigeonhole ereject vs reject
by Robert Blayzor 14 Jan '15

14 Jan '15

Currently pigeonhole supports reject which would generate a NDR for each message. (If I understand the current documentation) Using Dovecot LMTP it would be more optimal to kick a 5xx back to the primary MTA to reject the delivery rather than generating more back scatter NDRs. Anyone know if this is possible now (to forgo the NDR)'s or do we need to wait for ereject ? If so, how much work is involved? -- Robert inoc.net!rblayzor http://inoc.net/

2 7

blowfish crypt
by Edgar Pettijohn 14 Jan '15

14 Jan '15

I'm setting up a new server and am planning on using blowfish crypt. This page from the wiki http://wiki2.dovecot.org/Authentication/PasswordSchemes says the password will begin with $2a$, however mine are starting $2b$. Is this wrong on the wiki or am I doing something wrong? Just want to catch this before it becomes a problem. Thanks, Edgar

1 1

dovecot and glusterfs
by Michael Schwartzkopff 13 Jan '15

13 Jan '15

Hi, I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files. With the help of the available documentation about NFS and fcntl locks I managed to find out the following: With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless. With mdbox and maildir format I could reliably crash my mail storage just by delivering mails to the both dovecots via LMTP to the same user. In maildir dovecot seems not the set / respect the fnctl locks of the index file. dotlocks do not seems to work either with mdbox. So I think the only solution os to use a director in a real world setup. Or is there any non-obvious trick that I did not check? I think it would be really nice if dovecot could use glusterfs for realy scalable postbox servers. Mit freundlichen Grüßen, Michael Schwartzkopff -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

4 6

Proxying when src and dest on non standard ports
by Thomas HUMMEL 13 Jan '15

13 Jan '15

Hello, I want to test dovecot proxying, on a user basis, in a simple set up (proxy, no proxy_maybe and forward the passwd to the remote server). This seem pretty simple except that, in order to test it alongside the production server, I set up 3 servers : . the proxy, listening to pop and imap on non standard ports 9143, 9993, 9110, 9995 . a proxy destination for some users, listening on 10143, 10993, 109110, 10995 . another proxy destination for some other users, listening on 20145, 20993, 209110, 20995 But there seems to be only one 'port' extra_field, so how can I handle the proxying of both IMAP and POP3 (especially for the same user who can use both) ? Thanks. -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Groupe Exploitation et Infrastructure

1 0

Re: dovecot-2.2.15 process dovecot/imap segfaults when dict process has troubles
by Marcin Mirosław 13 Jan '15

13 Jan '15

W dniu 13.01.2015 o 13:53, Lazy pisze: > I can't reproduce this with redis dict. > > Could You provide us with a doveconf -n ? > Does this also happen with other dict engines ? Working configuration: auth_cache_size = 64 k auth_cache_ttl = 10 mins auth_master_user_separator = * auth_mechanisms = plain login deliver_log_format = msgid=%m: from=%f: phys=%p: virt=%w %$ dict { domainquota = pgsql:/etc/dovecot/dovecot-dict-sql-domain.conf.ext lastlogin = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 999 first_valid_uid = 999 last_valid_gid = 999 last_valid_uid = 999 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * mail_gid = 999 mail_location = maildir:~/.maildir:INDEX=/data/poczta/indeksy/%d/%n mail_log_prefix = "%s(%u) <%{session}>: " mail_plugins = acl fts fts_squat listescape quota notify mail_log stats zlib mail_temp_dir = /data/poczta/tmp mail_uid = 999 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include vari ables body enotify environment mailbox date ihave duplicate namespace { location = maildir:%%h/.maildir:INDEX=/data/poczta/indeksy/shared/%d/%n prefix = shared/%%n/ type = shared } namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Archives { special_use = \Archive } mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox Wiadomości-śmieci { special_use = \Junk } mailbox Wiadomo&AVs-ci-&AVs-mieci { special_use = \Junk } prefix = separator = / } passdb { driver = pam master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/data/poczta/acl/global-acls:cache_secs=60 acl_shared_dict = file:/data/poczta/acl/shared-mailboxes.db fts = squat fts_squat = partial=4 full=10 last_login_dict = proxy::lastlogin last_login_key = lastlogin/%u/%s/%r listescape_char = \ quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = file:~/sieve;active=~/.dovecot.sieve sieve_default = /etc/dovecot/sieve/sieve_default_spam-to-junk.sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 1M stats_refresh = 30 s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmaster(a)example.com protocols = imap pop3 lmtp sieve quota_full_tempfail = yes service auth-worker { user = root } service auth { unix_listener auth-client { group = mail mode = 0660 } unix_listener auth-userdb { mode = 0777 } } service dict { unix_listener dict { group = poczta mode = 0660 } } service stats { fifo_listener stats-mail { group = poczta mode = 0660 } } ssl_cert = </etc/ssl/private/example.com.crt ssl_cipher_list = DEFAULT:!EXPORT:!LOW:!MEDIUM:!MD5 ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/example.com.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = acl fts fts_squat listescape quota notify mail_log stats zlib sieve } protocol lda { mail_plugins = quota sieve } protocol imap { mail_max_userip_connections = 25 mail_plugins = acl fts fts_squat listescape quota notify mail_log stats zlib imap_acl imap_quota imap_stats imap_zlib last_login } protocol pop3 { mail_plugins = acl fts fts_squat listescape quota notify mail_log stats zlib last_login } Then I'm adding: quota = dict:Quota:%d:proxy::domainquota and I'm doing: chmod 000 /etc/dovecot/dovecot-dict-sql-domain.conf.ext Next I'm restarting dovecot, trying to use IMAP account (reading, removing emails etc.) and from time to time dovecot/imap throws core. Marcin

1 0

Moving or "upgrading" from MD5 to SSHA512
by Jyri Hovila [Turvamies.fi] 13 Jan '15

13 Jan '15

Hello, world! I have a long-running Dovecot & Postfix installation using PostgreSQL back-end. Until now I've been using MD5 hashing but would like to "upgrade" to the salted SSHA512. Is there a way to configure Dovecot so that it would automatically detect the type of the hash stored in the database, so that users who have changed their password (and thus being hashed with SHA512) and users still having an MD5 hash would both be able to authenticate at the same time? Yours, Jyri -- +358-50-5632104 (24/7) +358-46-8822157 (backup) jyri.hovila(a)turvamies.fi

3 3

dovecot-2.2.15 process dovecot/imap segfaults when dict process has troubles
by Marcin Mirosław 13 Jan '15

13 Jan '15

Hi! I hope it's not too late to wish all the best in 2015 year! I'm writing the email because I've noticed that if dict process has any troubles with configuration then dovecot/imap throws segfault: log: Jan 12 16:51:59 mail dovecot: dict: Error: Failed to initialize dictionary 'lastlogin': dict pgsql: Can't open configuration file /etc/dovecot/dovecot-dict-sql.conf.ext: Permission denied Jan 12 16:51:59 mail dovecot: dict: Error: Failed to initialize dictionary 'lastlogin': dict pgsql: Can't open configuration file /etc/dovecot/dovecot-dict-sql.conf.ext: Permission denied Jan 12 16:52:00 mail dovecot: master: Error: service(imap): child 24243 killed with signal 11 (core dumped) and here is backtrace: (gdb) bt #0 io_remove_full (_io=0x7fb02b0afce0, closed=false) at ioloop.c:119 #1 0x00007fb02aac5f40 in io_remove (io=<optimized out>) at ioloop.c:154 #2 0x00007fb02aa841b2 in client_dict_finish_transaction (dict=dict@entry=0x7fb02b0afc20, id=<optimized out>, ret=ret@entry=-1) at dict-client.c:274 #3 0x00007fb02aa84286 in client_dict_disconnect (dict=dict@entry=0x7fb02b0afc20) at dict-client.c:459 #4 0x00007fb02aa8493d in client_dict_deinit (_dict=0x7fb02b0afc20) at dict-client.c:519 #5 0x00007fb02aa28ece in last_login_dict_deinit (user=<optimized out>) at last-login-plugin.c:34 #6 0x00007fb02aa28f92 in last_login_user_deinit (user=0x7fb02b0ab400) at last-login-plugin.c:46 #7 0x00007fb02ab820bf in mail_user_unref (_user=_user@entry=0x7fb02b0bbdc8) at mail-user.c:163 #8 0x00007fb02ac1ee0d in client_default_destroy (client=0x7fb02b0bbd50, reason=<optimized out>) at imap-client.c:284 #9 0x00007fb02ac29549 in client_kill_idle (client=0x7fb02b0bbd50) at main.c:105 #10 0x00007fb02ac296a5 in client_kill_idle (client=0x7fb02b0bbd50) at main.c:129 #11 imap_die () at main.c:121 #12 0x00007fb02aac661d in io_loop_call_io (io=0x7fb02b093ce0) at ioloop.c:498 #13 0x00007fb02aac7a15 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x7fb02b091740) at ioloop-epoll.c:220 #14 0x00007fb02aac66d5 in io_loop_handler_run (ioloop=ioloop@entry=0x7fb02b091740) at ioloop.c:545 #15 0x00007fb02aac677a in io_loop_run (ioloop=0x7fb02b091740) at ioloop.c:522 #16 0x00007fb02aa5b5aa in master_service_run (service=0x7fb02b0915d0, callback=<optimized out>) at master-service.c:566 #17 0x00007fb02ac116cd in main (argc=<optimized out>, argv=<optimized out>) at main.c:412 Marcin

1 0