May 2014 - dovecot - dovecot.org

[Dovecot] Some Information about compression rates to expect using zlib/xz compression
by Robert Nowotny 07 May '14

07 May '14

compression : xz, Level 6 here some compression rates I experienced : Diskspace usage : user1: uncompressed : 2.3GB --> compressed 1 GB user2: uncompressed : 6.2GB --> compressed 3.9 GB just for your reference Ing. Robert Nowotny Rotek GmbH Vienna/Austria

1 0

Re: [Dovecot] LMTP, TLS/SSL, authentication, proxy
by Stanislas SABATIER 06 May '14

06 May '14

Hello, unfortunatly, TLS doesn't seem to work on LMTP, anyway, not by simply put ssl=yet to the lmtp inet_listener. I've tried and when I *openssl s_client -connect* to the lmtp listener I've got the following response : CONNECTED(00000003) 34379118248:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s23_clnt.c:766: My postfix server can't neither establish a TLS connection on LMTP listener. On 22.10.2013, at 14.37, Jogi Hofmüller <jogi(a)mur.at> wrote: Can dovecot use TLS/SSL on LTMP inet socket? Oct 26, 2013 8:11:19 am , Timo Sirainen <tss(a)iki.fi> replied: Probably. Try adding ssl=yes to the inet_listener {}. --------------------------------------------------------------- Stanislas SABATIER s.sabatier(a)pobox.com ---------------------------------------------------------------

1 0

[Dovecot] Status of sieve-extdata?
by Jiri Bourek 06 May '14

06 May '14

Hi, I'd like to ask about status of extdata plugin for sieve. The wiki page (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extdata) mentions versions for PigeonHole 0.2 and 0.3, but there seems to be no version for 0.4 and Dovecot 2.2. Is the plugin dead or is it planned to make version for Dovecot 2.2 (when time allows I guess)? Regards J.

2 5

[Dovecot] on High Load using IMAPSYNC : Panic: file ostream-lzma.c: line 147: unreached. Dovecot 2.2.12 with zlib/XZ compression
by Robert Nowotny 06 May '14

06 May '14

when syncing Mailboxes with IMAPSYNC, using xz compression the dbox file gets corrupted. the same configuration was working ok on low load with xz compression. Dovecot Version 2.2.12 Linux Mint 16 EXT4 Filesystem another dovecot user already reported the same problem on the mailinglist, using dsync. That user switched to bz2 compression what worked for him. the answer was : > are you on latest patchlevel ? > there are some dsync fixes since 2.2.12 but it is not a problem of dsync. when looking at the code, we get here an unexpected returnvalue when flushing. I propose it would be good practice to write some logging in the default switch cases, to see the unexpected returnvalues in the errorlog. something like : ostream-lzma.c : -- Line 147 : i_unreached(); ++ Line 147 : i_fatal("unexpected lzma errorcode when flushing : %s", ret); I will try to change it and let You know. maybe You can also investigate and let me know when fixed. Here the log : May 5 14:53:10 vm-imap dovecot: imap(rrernboeck): Panic: file ostream-lzma.c: line 147: unreached May 5 14:53:10 vm-imap dovecot: imap(rrernboeck): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x67450) [0x7ff99d8ce450] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x6752e) [0x7ff99d8ce52e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ff99d88955e] -> /usr/local/lib/dovecot/lib20_zlib_plugin.so(+0x4f0a) [0x7ff99ce87f0a] -> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_flush+0x3e) [0x7ff99d8e6dbe] -> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_nflush+0xf) [0x7ff99d8e71df] -> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_nfinish+0x9) [0x7ff99d8e7229] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(dbox_save_end+0x29) [0x7ff99db7cfa9] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mdbox_save_finish+0x39) [0x7ff99db76619] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_finish+0x4d) [0x7ff99dbb6a4d] -> dovecot/imap [rrernboeck 127.0.0.1 APPEND]() [0x40be09] -> dovecot/imap [rrernboeck 127.0.0.1 APPEND](command_exec+0x3c) [0x416cfc] -> dovecot/imap [rrernboeck 127.0.0.1 APPEND]() [0x40b491] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7ff99d8de527] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7ff99d8df2b7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ff99d8de0c8] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ff99d88e753] -> dovecot/imap [rrernboeck 127.0.0.1 APPEND](main+0x2af) [0x40b06f] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7ff99d4c0de5] -> dovecot/imap [rrernboeck 127.0.0.1 APPEND]() [0x40b1cc] May 5 14:53:10 vm-imap dovecot: imap(rrernboeck): Fatal: master: service(imap): child 47957 killed with signal 6 (core dumped) May 5 14:53:10 vm-imap dovecot: imap(rrernboeck): Error: Corrupted dbox file /home/vmail/virtualmailboxes/rrernboeck/storage/m.1 (around offset=1302): msg header has bad magic value

1 0

[Dovecot] doveadm auth and the "nologin" extra field
by Axel Luttgens 05 May '14

05 May '14

Hello, Still busy with details... Considering, as in my previous example, a password_query returning '!' or NULL for the "nologin" column, depending on an account's status (suspended or not). Let's consider a suspended user "some.user". In the case of a successful authentication, one has: sh-3.2# doveadm auth test some.user goodpassword; echo $? passdb: some.user auth succeeded extra fields: user=some.user nologin 0 On the other hand, in the case of an authentication failure: sh-3.2# doveadm auth test some.user badpassword; echo $? passdb: some.user auth failed extra fields: user=some.user nologin=! 77 So, this is similar to what happens in a connection (pop3, imap...): when present, the nologin info is always taken into account, even in the case of an authentication failure. Again, this may raise some concerns about the consistency of such a behavior. Is this guaranteed to always behave that way, because of some rationale I'm currently missing, or does it go about some overlooked combination, liable to be inadvertently "corrected" in the future? I haven't been able to find a definitive answer in the wiki or in the code about such matters. This is particularly important in the case of doveadm, since its output requires parsing for extracting such informations (the exit code alone isn't sufficient); should above behavior be changed without notice, and a script could suddenly take the worst decisions... BTW, why: nologin in the first output, and: nologin=! in the second output? TIA, Axel

2 2

[Dovecot] Segfault in dovecot-lda when resolver is unavailable
by Marcin Mirosław 05 May '14

05 May '14

Hi list! Recently I noticed that dovecot-lda throws segfault when resolver is unavailable and with imapc configured. $ cat /etc/resolv.conf search . nameserver 127.0.0.1 backtrace: # gdb -q /usr/libexec/dovecot/dovecot-lda 'core-1398282784-8-11-!usr!libexec!dovecot!dovecot-lda-8805' [...] Core was generated by `/usr/libexec/dovecot/deliver -d marcin(a)mejor.pl'. Program terminated with signal 11, Segmentation fault. #0 0x000002b612afaf72 in _int_free (av=0x2b612e215c0 <main_arena>, p=0x1b23bfabe0, have_lock=0) at malloc.c:3903 3903 unlink(p, bck, fwd); (gdb) bt #0 0x000002b612afaf72 in _int_free (av=0x2b612e215c0 <main_arena>, p=0x1b23bfabe0, have_lock=0) at malloc.c:3903 #1 0x000002b612e96a4e in buffer_free (_buf=_buf@entry=0x1b23bfa948) at buffer.c:144 #2 0x000002b612ebeca8 in array_free_i (array=0x1b23bfa948) at array.h:108 #3 priorityq_deinit (_pq=_pq@entry=0x1b23bfabb0) at priorityq.c:38 #4 0x000002b612eafa57 in io_loop_destroy (_ioloop=_ioloop@entry=0x395a4da91d0) at ioloop.c:495 #5 0x000002b61306f225 in imapc_client_run_post (client=<optimized out>) at imapc-client.c:175 #6 imapc_client_run (client=0x1b23bda930) at imapc-client.c:181 #7 0x000002b613067158 in imapc_list_try_get_root_sep (list=0x1b23bd7b60, sep_r=sep_r@entry=0x395a4da9247 "") at imapc-list.c:297 #8 0x000002b6130671ae in imapc_list_get_hierarchy_sep (_list=<optimized out>) at imapc-list.c:310 #9 0x000002b613065d9f in imapc_list_get_fs_name (list=0x1b23bd7b60, name=0x0) at imapc-list.c:396 #10 0x000002b613066031 in imapc_list_get_path (_list=0x1b23bd7b60, name=0x0, type=MAILBOX_LIST_PATH_TYPE_MAILBOX, path_r=0x395a4da9338) at imapc-list.c:437 #11 0x000002b613089785 in mailbox_list_get_root_path (list=<optimized out>, type=type@entry=MAILBOX_LIST_PATH_TYPE_MAILBOX, path_r=path_r@entry=0x395a4da9338) at mailbox-list.c:1276 #12 0x000002b612ff8eb5 in quota_add_user_namespace (quota=0x1b23bcea80, ns=0x1b23bd7a10) at quota.c:745 #13 0x000002b61300073a in quota_mailbox_list_created (list=0x1b23bfd9c0) at quota-storage.c:590 #14 0x000002b613084581 in hook_mailbox_list_created (list=list@entry=0x1b23bfd9c0) at mail-storage-hooks.c:328 #15 0x000002b61308836f in mailbox_list_create (driver=driver@entry=0x2b6130e677b "maildir++", ns=0x1b23bd7a10, set=set@entry=0x395a4da94a0, flags=<optimized out>, list_r=list_r@entry=0x1b23bd7d60, error_r=error_r@entry=0x395a4da9498) at mailbox-list.c:210 #16 0x000002b613065cf6 in imapc_list_get_fs (list=list@entry=0x1b23bd7b60) at imapc-list.c:379 #17 0x000002b61306601e in imapc_list_get_path (_list=0x1b23bd7b60, name=0x0, type=MAILBOX_LIST_PATH_TYPE_MAILBOX, path_r=0x395a4da9588) at imapc-list.c:433 #18 0x000002b613089785 in mailbox_list_get_root_path (list=<optimized out>, type=type@entry=MAILBOX_LIST_PATH_TYPE_MAILBOX, path_r=path_r@entry=0x395a4da9588) at mailbox-list.c:1276 #19 0x000002b612ff8eb5 in quota_add_user_namespace (quota=0x1b23bcea80, ns=0x1b23bd7a10) at quota.c:745 #20 0x000002b61300073a in quota_mailbox_list_created (list=0x1b23bd7b60) at quota-storage.c:590 #21 0x000002b613084581 in hook_mailbox_list_created (list=list@entry=0x1b23bd7b60) at mail-storage-hooks.c:328 #22 0x000002b61308836f in mailbox_list_create (driver=driver@entry=0x2b6130e6d1b "imapc", ns=ns@entry=0x1b23bd7a10, set=set@entry=0x395a4da9720, flags=<optimized out>, list_r=list_r@entry=0x395a4da9708, error_r=error_r@entry=0x395a4da9810) at mailbox-list.c:210 #23 0x000002b61307f500 in mail_storage_create_full (ns=ns@entry=0x1b23bd7a10, driver=<optimized out>, driver@entry=0x0, data=<optimized out>, flags=flags@entry=(unknown: 0), storage_r=storage_r@entry=0x395a4da97c0, error_r=error_r@entry=0x395a4da9810) at mail-storage.c:356 #24 0x000002b61307f7cc in mail_storage_create (ns=ns@entry=0x1b23bd7a10, driver=driver@entry=0x0, flags=flags@entry=(unknown: 0), error_r=error_r@entry=0x395a4da9810) at mail-storage.c:407 #25 0x000002b613077a48 in namespace_add (user=user@entry=0x1b23bcf7e0, ns_set=0x1b23bd0db8, unexpanded_ns_set=0x1b23bcff30, mail_set=mail_set@entry=0x1b23bd07e8, ns_p=ns_p@entry=0x1b23bd3b00, error_r=error_r@entry=0x395a4da9950) at mail-namespace.c:170 #26 0x000002b613078aaf in mail_namespaces_init (user=0x1b23bcf7e0, error_r=error_r@entry=0x395a4da9950) at mail-namespace.c:344 #27 0x000002b6130901bf in mail_storage_service_init_post (error_r=0x395a4da9950, mail_user_r=0x395a4da9b18, priv=0x395a4da9960, user=0x1b23bbde50, ctx=<optimized out>) at mail-storage-service.c:685 #28 mail_storage_service_next (ctx=ctx@entry=0x1b23bbd4f0, user=0x1b23bbde50, mail_user_r=mail_user_r@entry=0x395a4da9b18) at mail-storage-service.c:1223 #29 0x000002b6130904b5 in mail_storage_service_lookup_next (ctx=0x1b23bbd4f0, input=input@entry=0x395a4da9b80, user_r=user_r@entry=0x395a4da9a70, mail_user_r=mail_user_r@entry=0x395a4da9b18, error_r=error_r@entry=0x395a4da9a60) at mail-storage-service.c:1258 #30 0x0000001b22341ccb in main (argc=3, argv=0x1b23bbb060) at main.c:412 # doveconf -n # 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.13.8-hardened x86_64 Gentoo Base System release 2.2 auth_cache_size = 1 k auth_master_user_separator = * auth_mechanisms = login digest-md5 cram-md5 plain deliver_log_format = msgid=%m: from=%f: phys=%p: virt=%w %$ dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_uid = 8 last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_attachment_dir = /dane/domeny/zalaczniki mail_attachment_min_size = 10000 k mail_cache_min_mail_count = 20 mail_gid = mail mail_log_prefix = "%s(%u) <%{session}>: " mail_plugins = quota notify mail_log stats zlib mail_privileged_group = mail mail_uid = mail maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy inc$ ude variables body enotify environment mailbox date ihave mdbox_preallocate_space = yes mdbox_rotate_interval = 60 days mdbox_rotate_size = 50 M namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . subscriptions = yes } passdb { args = /etc/dovecot/.master.user driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 1M stats_refresh = 30 s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours zlib_save = bz2 zlib_save_level = 2 } postmaster_address = postmaster(a)mejor.pl protocols = imap pop3 sieve service auth { unix_listener auth-userdb { group = mail mode = 0660 user = root } } service dict { unix_listener dict { user = mail } } service stats { fifo_listener stats-mail { mode = 0600 user = mail } } ssl_cert = </etc/ssl/gwiazdka.mejor.pl.pem ssl_key = </etc/ssl/gwiazdka.mejor.pl.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lda { mail_plugins = quota notify mail_log stats zlib sieve } protocol imap { mail_max_userip_connections = 25 mail_plugins = quota notify mail_log stats zlib imap_quota imap_stats imap_zlib } Thanks, Marcin

2 2

[Dovecot] %{orig_user} missing in checkpassword-Script
by dovecot.pkoch＠dfgh.net 05 May '14

05 May '14

Dear dovecot maintainers: I'm using SSL client certificates together with a checkpassword scripts to authenticate our users. My problem is: In the checkpassword script the AUTH_USER environment variable will either contain the username that was configured in the mailclient (if auth_ssl_username_from_cert=false) or the username from the certificate (if auth_ssl_username_from_cert=true). I would like to compare both values, i.e. the %{user} Dovecot-variable and the %{orig_user} Dovecot-variable. But the environment of a checkpassword-script has only one of them. I tried myself and found the following: - the environment of a checkpassword script is setup by checkpassword_setup_env() in db-checkpassword.c - checkpassword_setup_env() calls env_put_auth_vars() - env_put_auth_vars() creates AUTH_xxx environment variables for all entries of the auth_request_get_var_expand_table() - the auth_request_get_var_expand_table_full() routine does not contain the original user, but the auth_request-struct does. So I changed the dovecot sourcecode (version 2.2.12) as follows In src/auth/auth-request.h line 152 I replaced #define AUTH_REQUEST_VAR_TAB_COUNT 27 by #define AUTH_REQUEST_VAR_TAB_COUNT 30 In src/auth/auth-request.c around line 2027 I replaced the following lines at the end of auth_request_var_expand_static_tab { '\0', NULL, "session_pid" }, /* be sure to update AUTH_REQUEST_VAR_TAB_COUNT */ { '\0', NULL, NULL } }; by { '\0', NULL, "session_pid" }, { '\0', NULL, "orig_user" }, { '\0', NULL, "orig_username" }, { '\0', NULL, "orig_domain" }, /* be sure to update AUTH_REQUEST_VAR_TAB_COUNT */ { '\0', NULL, NULL } }; In src/auth/auth-request.c around line 2116 I replaced the following lines at the end of function auth_request_get_var_expand_table_full() tab[26].value = auth_request->session_pid == (pid_t)-1 ? NULL : dec2str(auth_request->session_pid); return ret_tab; by tab[26].value = auth_request->session_pid == (pid_t)-1 ? NULL : dec2str(auth_request->session_pid); if (auth_request->original_username != NULL) { tab[27].value = escape_func(auth_request->original_username, auth_request); tab[28].value = escape_func(t_strcut(auth_request->original_username, '@'), auth_request); tab[29].value = strchr(auth_request->original_username, '@'); if (tab[29].value != NULL) { tab[29].value = escape_func(tab[29].value+1, auth_request); } } return ret_tab; This will add AUTH_ORIG_USER, AUTH_ORIG_USERNAME and AUTH_ORIG_DOMAIN environment variables to the environment of every checkpassword script. If this is the correct way to extend the environment of a chackpassword-script then you might consider adding these minor changes to the dovecot-source. Kind regards and thanks very much for this wonderful project Peter Koch

2 1

[Dovecot] CONTEXT=SORT
by Michael M Slusarz 05 May '14

05 May '14

Any reason this is not listed in the CAPABILITY string? I see it listed as a TODO here: http://wiki2.dovecot.org/Roadmap But it seems to work fine (as of 2.2.12): 1 UID SORT RETURN (CONTEXT COUNT) (SUBJECT) UTF-8 UNDELETED * ESEARCH (TAG "1") UID COUNT 12 1 OK Sort completed (0.000 secs). 2 UID SORT RETURN (UPDATE COUNT) (SUBJECT) UTF-8 UNDELETED * ESEARCH (TAG "2") UID COUNT 12 2 OK Sort completed (0.000 secs). [Add message to mailbox] 3 NOOP * 13 EXISTS * 1 RECENT * ESEARCH (TAG "2") UID ADDTO (0 62475) 3 OK NOOP completed. 4 CANCELUPDATE "2" 4 OK Updates cancelled. 5 UID SORT RETURN (PARTIAL 1:10) (SUBJECT) UTF-8 UNDELETED * ESEARCH (TAG "5") UID PARTIAL (1:10 NIL) 5 OK Sort completed (0.000 secs). michael

2 2

[Dovecot] Dovecot proxy
by Alex Ferrara 05 May '14

05 May '14

Hi everyone, I have a problem that hopefully has an easy solution. I am setting up an IMAP proxy in a DMZ network. It will connect to the real IMAP server and authenticate using "driver = imap", and this I have working really nicely. What I want to do is have it look up a list of users that are allowed to connect through the proxy before proxying the connection, as not all users with an account are permitted to access their email from the internet. I thought that using a post-login script would get me out of trouble, but it isn't possible in a relay configuration. dovecot.conf ## Dovecot configuration file mail_uid = dovecot mail_gid = dovecot protocols = imap listen = *, :: passdb { driver = imap # IMAP server to authenticate against args = host=192.168.1.1 # IMAP server to connect to for mailbox default_fields = proxy=yes host=192.168.1.1 } userdb { driver = prefetch } auth_mechanisms = plain login # This is the auth service used by Postfix to do dovecot auth. service auth { unix_listener auth-userdb { } inet_listener { port = 12345 } } ## ## SSL settings ## # These will need to ba adjusted to point to *your* certificates, not mine 8-) # The ssl_ca line refers to the intermediate certificate bundle which may or may not be required by your SSL provider ssl_cert = </etc/ssl/certs/mail.domain.com.au.pem ssl_key = </etc/ssl/private/mail.domain.com.au.key #ssl_ca = </etc/pki/tls/certs/ca.crt ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL

2 2

[Dovecot] Dsyncing mail in director setup
by Murray Trainer 05 May '14

05 May '14

Hi All, I have a pair of dovecot director proxies and six dovecot backend mailstores using NFS v4.1 to access five filesystems on EMC VNX NFS storage. This is all working fine until I try and dsync new email mailboxes via one of the mailstores onto the NFS storage. I get major NFS lockups at random on one or more of the mailstores with none of the NFS mounts accessible on one or more of the mailstores. I am getting our storage guy to look into it to see if it is just a performance issue. I can understand there may be locking issues but I wouldn't have thought it would have this effect. I also wonderif there is a better way to run dsync so it handle writing to NFS? Maybe passing it some of the nfs related dovecot options? Thanks Murray

1 0