- dovecot - dovecot.org

Re: Problem with Let's Encrypt Certificate
by Joseph Tam 23 Feb '17

23 Feb '17

yacinechaouche(a)yahoo.com writes: > Interesting. Is there any particular benefit in having only one file > for both certificate and private key ? I find that putting private key > in a separate file feels more secure. It's convenient to have key and cert in one place if you don't need the certificate to be publically readable. Keeping it in separate files would add slightly more security (defense in depth), that would protect from, for example, an admin fumble or bug in the SSL library. "Michael A. Peters" <mpeters(a)domblogger.net> writes: >> I use dehydrated (with Cloudflare DNS challenges) and as far as I know, >> it seems to generate a new private key every time. > > Yeah that would be a problem for me because I implement DANE. It's on my to-do list, but I think you can use dehydrated in signing mode. --signcsr (-s) path/to/csr.pem Sign a given CSR, output CRT on stdout (advanced usage) In this way, you can reuse private key, as well as making it more secure by removing a privileged operations (private key acces) allowing dehydrated to be run as a non-privilged/separate user. Joseph Tam <jtam.home(a)gmail.com>

2 2

Pigeonhole External pipe script going zombie?
by Matt Simpson 23 Feb '17

23 Feb '17

I’m running dovecot 2.2.27 and pigeonhole 0.4.16 on FreeBSD 11. I’m using the pigeonhole/sieve external pipe plugin to run a Perl program to send a Pushover notification when certain messages are received. The Perl script is executed, and the notification is sent. But then the script task seems to go zombie until it is killed after a timeout. In the user’s sieve log, I get a message like error: msgid=<20170112191921.66140.qmail(a)v1.redhorse.me>: pipe action: failed to pipe message to program `sievepush.pl': refer to server log for more information. [2017-01-12 14:19:36]. (even though the message really was piped to the program successfully) In the dovecot server log, I see Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: sieve: Executing script from `/usr/home/matt/maildoms/.dovecot.svbin' Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: sieve: action pipe: running program: sievepush.pl Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: Mailbox stdin: Opened mail UID=1 because: mail stream Jan 12 14:19:21 v1 dovecot: lda(matt): Debug: waiting for program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl' to finish after 0 msecs Jan 12 14:19:31 v1 dovecot: lda(matt): Debug: program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl'(66145) execution timed out after 10000 milliseconds: sending TERM signal Jan 12 14:19:36 v1 dovecot: lda(matt): Debug: program `/usr/local/lib/dovecot/sieve-pipe/sievepush.pl' (66145) did not die after 5000 milliseconds: sending KILL signal In the process list during that 10 second interval, I see matt 66142 29972 801 801 0 S - 0:00.00 bin/qmail-local -- matt /home/matt/maildoms jmn-matt - jmn-m matt 66143 66142 801 801 0 S - 0:00.00 /var/qmail/bin/preline -f /usr/local/libexec/dovecot/dovecot matt 66144 66143 801 801 0 S - 0:00.01 /usr/local/libexec/dovecot/dovecot-lda matt 66145 66144 801 801 0 Z - 0:00.65 <defunct> I’m not a Unix programming ace, but from what I’ve been able to find out, this seems to mean that the lda process is forking another process to run the pipe script, and not getting the proper notification when it finishes (not issuing a wait?). So after 10 seconds, it sends a TERM to the task which is no longer running, and when that doesn’t work, it sends a KILL. Anybody know what’s happening here? doveconf -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 11.0-RELEASE-p2 amd64 auth_verbose = yes default_vsz_limit = 128 M lock_method = flock mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = prefix = } passdb { args = imap driver = pam } plugin { recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve-pipe sieve_pipe_exec_timeout = 10s sieve_plugins = sieve_extprograms } protocols = imap service auth { unix_listener auth-master { group = qnofiles mode = 0660 user = alias } user = root } service imap-login { process_min_avail = 3 vsz_limit = 94 M } ssl_cert = </usr/local/etc/letsencrypt/live/dummy.redhorse.me/cert.pem ssl_key = # hidden, use -P to show it syslog_facility = local0 userdb { driver = passwd } verbose_proctitle = yes protocol lda { mail_plugins = " sieve" }

2 3

How to add Ubuntu 16.04.2 server to join and authenticate with domain controller windows2012r2
by Basdove 23 Feb '17

23 Feb '17

Hi, 1. I have Ubuntu 16.0.4.2 server edition 2. I have windows server 2012R2 domain controller (Active directory installed) 3. I want to join and Authenticate Ubuntu with windows server 2012 domain controller 4. Please provide the clean document Thank you, Baskaranand

3 2

imap/pop3-login assertion failed in dovecot 2.2.27 (c0f36b0)
by Edgaras Lukoševičius 23 Feb '17

23 Feb '17

Hello, I have noticed few errors like this. I don't recall seeing that with version 2.2.10 Feb 23 05:20:03 mail21 dovecot[6569]: imap-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 05:50:03 mail21 dovecot[15044]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 05:50:03 mail21 dovecot[15044]: imap-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 06:30:04 mail21 dovecot[13266]: imap-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 06:30:04 mail21 dovecot[13266]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 07:10:04 mail21 dovecot[14298]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 07:10:04 mail21 dovecot[14298]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 07:10:04 mail21 dovecot[14298]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0) Feb 23 08:00:04 mail21 dovecot[13138]: pop3-login: Panic: file login-proxy-state.c: line 77 (login_proxy_state_deinit): assertion failed: (rec->num_waiting_connections == 0)

1 0

Problem with Let's Encrypt Certificate
by Bastian Sebode 23 Feb '17

23 Feb '17

Hello Folks, my StartCom SSL-Certificate expires soon and so I wanted to switch to Let's Encrypt Certificates instead. Unfortunatelly Thunderbird seems not to like it, although all -tested- other Clients work without any problems. When I connect with Thunderbird it sends an "Encrypted Alert" directly after the TLS handshake although Dovecot wants to continue the session. In the Dovecot Log it says: Feb 17 17:27:17 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [82.100.242.26] Feb 17 17:27:17 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [82.100.242.26] Feb 17 17:27:17 imap-login: Warning: SSL alert: where=0x4004, ret=554: fatal bad certificate [82.100.242.26] But the certificate is okay, cause it works with other Mailclients and openssl also says so. What certificate is Thunderbird complaining about? Thunderbird says something like "There's no supported authentication method". I don't use any Certificates for Client Authentication, neither in Dovecot nor in Thunderbird. When I do, it fails the same way. Weirdly my friend uses the same Dovecot Version with Let's Encrypt on his Server and it works with Thunderbird without any flaws. Mine fails the same way in his Thunderbird and also in a fresh installation. After two weeks of investigating I still have no clue why it behaves like this. I uploaded two Wireshark tracefiles, further logs and dovecot -n, may be someone sees any possible reasons for this weird behavior or has any further tips on solving this issue. https://sebode-online.de/dovecot-letsencrypt/ Every hint is highly appreciated! Best Regards Bastian -- Bastian Sebode Fachinformatiker Systemintegration LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de LINET in den sozialen Netzwerken: www.twitter.com/linetservices | www.facebook.com/linetservices Wissenswertes aus der IT-Welt: www.linet-services.de/blog/ Geschäftsführung: Timo Springmann, Mirko Savic und Moritz Bunkus HR B 9170 Amtsgericht Braunschweig USt-IdNr. DE 259 526 516

15 27

Scaling to 10 Million IMAP sessions on a single server
by KT Walrus 22 Feb '17

22 Feb '17

I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-c… <https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-c…> about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? I think the current implementation of having a separate process manage each active IMAP session (w/ the possibility of moving idling sessions to a single hibernate process) will never be able to deploy a single server managing 10 Million IMAP sessions. But, would it be possible to implement a new IMAP server plugin that uses a fixed configurable pool of “worker” processes, much like NGINX or PHP-FPM does. These servers can probably scale to 10 Million TCP connections, if the server is carefully tuned and has enough cores/memory to support that many active sessions. I’m thinking that the new IMAP server could use some external database (e.g., Redis or Memcached) to save all the sessions state and have the “worker” processes poll the TCP sockets for new IMAP commands to process (fetching the session state from the external database when it has a command that is waiting on a response). The Dovecot IMAP proxies could even queue incoming commands to proxy many incoming requests to a smaller number of backend connections (like ProxySQL does for MySQL requests). That might allow each Dovecot proxy to support 10 Million IMAP sessions and a single backend could support multiple front end Dovecot proxies (to scale to 100 Million concurrent IMAP connections using 10 proxies for 100 Million connections and 1 backend server for 10 Million connections). Of course, the backend server may need to be beefy and have very fast NVMe SSDs for local storage, but changing the IMAP server to manage a pool of workers instead of requiring a process per active session, would allow bigger scale up and could save large sites a lot of money. Is this a good idea? Or, am I missing something? Kevin

8 15

v2.2.28 release candidate 2 released
by Timo Sirainen 22 Feb '17

22 Feb '17

http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc2.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc2.tar.gz.sig I'm assuming that most of the bugs are now found and fixed, so the final 2.2.28 should be out in a day or two. Changes since rc1: * Reverted [UNKNOWN-CTE] -> [PARSE] change + pop3c: Added pop3c_features=no-pipelining setting to prevent using PIPELINING extension even though it's advertised. - lmtp_user_concurrency_limit didn't work if userdb changed username - Fixed various bugs that were introduced in rc1 Here's the full list of changes again: * director: "doveadm director move" to same host now refreshes user's timeout. This allows keeping user constantly in the same backend by just periodically moving the user there. * When new mailbox is created, use initially INBOX's dovecot.index.cache caching decisions. * Expunging mails writes GUID to dovecot.index.log now only if the GUID is quickly available from index/cache. * pop3c: Increase timeout for PASS command to 5 minutes. * Mail access errors are no longer ignored when searching or sorting. With IMAP the untagged SEARCH/SORT reply is still sent the same as before, but NO reply is returned instead of OK. + Make dovecot.list.index's filename configurable. This is needed when there are multiple namespaces pointing to the same mail root (e.g. lazy_expunge namespace for mdbox). + Add size.virtual to dovecot.index when folder vsizes are accessed (e.g. quota=count). This is mainly a workaround to avoid slow quota recalculation performance when message sizes get lost from dovecot.index.cache due to corruption or some other reason. + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them in lib-dsasl for client side. + auth: Support filtering by SASL mechanism: passdb { mechanisms } + Shrink the mail processes' memory usage by not storing settings duplicated unnecessarily many times. + imap: Add imap_fetch_failure setting to control what happens when FETCH fails for some mails (see example-config). + imap: Include info about last command in disconnection log line. + imap: Created new SEARCH=X-MIMEPART extension. It's currently not advertised by default, since it's not fully implemented. + fts-solr: Add support for basic authentication. + Cassandra: Support automatically retrying failed queries if execution_retry_interval and execution_retry_times are set. + doveadm: Added "mailbox path" command. + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the wanted fields only if it doesn't require opening the email. + mail_vsize_bg_after_count setting added (see example-config). + mail_sort_max_read_count setting added (see example-config). + pop3c: Added pop3c_features=no-pipelining setting to prevent using PIPELINING extension even though it's advertised. - Index files: day_first_uid wasn't updated correctly since v2.2.26. This caused dovecot.index.cache to be non-optimal. - imap: SEARCH/SORT may have assert-crashed in client_check_command_hangs - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. - imap: Running time in tagged command reply was often wrongly 0. - search: Using NOT n:* or NOT UID n:* wasn't handled correctly - director: doveadm director kick was broken - director: Fix crash when using director_flush_socket - director: Fix some bugs when moving users between backends - imapc: Various error handling fixes and improvements - master: doveadm process status output had a lot of duplicates. - autoexpunge: If mailbox's rename timestamp is newer than mail's save-timestamp, use it instead. This is useful when autoexpunging e.g. Trash/* and an entire mailbox is deleted by renaming it under Trash to prevent it from being autoexpunged too early. - autoexpunge: Multiple processes may have been trying to expunge the same mails simultaneously. This was problematic especially with lazy_expunge plugin. - auth: %{passdb:*} was empty in auth-worker processes - auth-policy: hashed_password was always sent empty. - dict-sql: Merge multiple UPDATEs to a single statement if possible. - fts-solr: Escape {} chars when sending queries - fts: fts_autoindex_exclude = \Special-use caused crashes - doveadm-server: Fix leaks and other problems when process is reused for multiple requests (service_count != 1) - sdbox: Fix assert-crash on mailbox create race - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve was used. especially %{storage_id} was broken. - lmtp_user_concurrency_limit didn't work if userdb changed username

1 0

Re: How to dsync mdbox compressed to maildir uncompressed
by Daniel Betz 22 Feb '17

22 Feb '17

Hi Thomas, thank you for your help. This is very strange that it wont work here. can you doveconf -n the relevant parts especially mail_plugins= and plugin { } i think i have misconfigured something :( Regards, Daniel

2 1

Re: How to dsync mdbox compressed to maildir uncompressed
by Daniel Betz 22 Feb '17

22 Feb '17

Hi Thomas, this doesnt work: doveadm -o plugin/quota= -o plugin/zlib_save= backup -u imap at test.de maildir:~/Maildir file Maildir/cur/1487752497.M128284P11684.mail,S=7691,W=7887:2,S Maildir/cur/1487752497.M128284P11684.mail,S=7691,W=7887:2,S: gzip compressed data, from Unix Regards, Daniel

2 1

Sieve and multi-auth databases
by dovecot＠avv.solutions 21 Feb '17

21 Feb '17

Hello Community, I am currently facing the following: - dovecot+postfix+sieve are running smoothly using passwd-file authentication - if a add a second authentication scheme (let's say mysql), I face a problem with sieve: -- receiving thru postfix is ok on both passwd-file and mysql entries and correctly stored -- I am able to send from the server as before ** BUT sieve == does not authenticate anymore from a client (using the same configuration as before ie using imap credentials) == does not process the messages anymore Digging in the sieve logs, it reports not finding the scripts anymore for existing accounts found in the passwd-file Any idea? Thank you! - - - - + - - - - # Here is the 90-sieve.conf: plugin { sieve = file:/sd/MAIL_IMAP_POP/%d/%n/_dovecot-sieve;active=/sd/MAIL_IMAP_POP/%d/%n/_dovecot-sieve-active sieve_default = /sd/myhost/var/lib/dovecot/sieve/default.sieve sieve = /sd/MAIL_IMAP_POP/%d/%n/__Sieve sieve_global_dir = /sd/myhost/var/lib/dovecot/sieve/global/ sieve_before = /sd/MAIL_IMAP_POP/SieveBefore sieve_after = /sd/MAIL_IMAP_POP/%d/SieveAfter/ sieve_after2 = /sd/MAIL_IMAP_POP/SieveAfter/ sieve_plugins = sieve_extprograms sieve_extensions = +vnd.dovecot.filter sieve_filter_bin_dir = /etc/dovecot/sieve-filters } - - - - + - - - - # Authentication for SQL users. Included from 10-auth.conf. passdb sql { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext # Associated query: # password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; } userdb sql { driver = static args = uid=vmail gid=vmail home=/sd/MAIL_IMAP_POP/%d/%n:LAYOUT=fs }

2 2

segfault in lib20_expire_plugin
by Mario Arnold 21 Feb '17

21 Feb '17

Hello, after upgrade from [2.2.devel (34f7cc3)] to [2.2.devel (b3443fc)] dovecot stops with a segfault: Fatal: master: service(imap): child 21179 killed with signal 11 (core dumped) imap[21179]: segfault at 0 ip 00000000f726eef1 sp 00000000ffa3b050 error 4 in lib20_expire_plugin.so[f726d000+3000] gdb /usr/lib/dovecot/imap /var/_core/core_imap-11-5000-5000-21179 GNU gdb (Debian 7.12-6) 7.12.0.20161007-git Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug/.build-id/99/6f1cf1a262cf5738f075ec046d9a7d344d9693.debug...done. done. [New LWP 21179] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1". Core was generated by `dovecot/imap imap-postlogin'. Program terminated with signal SIGSEGV, Segmentation fault. #0 expire_mail_namespaces_created (ns=0xf814db90) at expire-plugin.c:428 428 expire-plugin.c: Datei oder Verzeichnis nicht gefunden. (gdb) bt full #0 expire_mail_namespaces_created (ns=0xf814db90) at expire-plugin.c:428 user = 0xf814d028 v = 0x0 db = 0xf815b960 error = 0xf81522c0 "Trash" #1 0xf75f9b21 in hook_mail_namespaces_created (namespaces=0xf8151008) at mail-storage-hooks.c:304 _data_stack_cur_id = 4 hooks__foreach_end = 0xf814dae8 hooks = 0xf814dad4 #2 0xf75ebf9f in mail_namespaces_init_finish (namespaces=0xf8151008, error_r=0xffa3b23c) at mail-namespace.c:383 _data_stack_cur_id = 3 ns = 0x0 prefixless_found = false __FUNCTION__ = "mail_namespaces_init_finish" #3 0xf75ec1bb in mail_namespaces_init (user=0xf814d028, error_r=0xffa3b23c) at mail-namespace.c:438 mail_set = 0xf814d118 ns_set = <optimized out> unexpanded_ns_set = <optimized out> namespaces = 0xf8151008 ns_p = <optimized out> i = <optimized out> count = <optimized out> count2 = <optimized out> __FUNCTION__ = "mail_namespaces_init" #4 0xf75fcd30 in mail_storage_service_init_post (ctx=<optimized out>, error_r=<optimized out>, mail_user_r=<optimized out>, priv=<optimized out>, user=<optimized out>) at mail-storage-service.c:728 mail_set = 0xf814d118 mail_user = 0xf814d028 #5 mail_storage_service_next_real (mail_user_r=<optimized out>, user=<optimized out>, ctx=<optimized out>) at mail-storage-service.c:1426 len = 4162116496 priv = {uid = 5000, gid = 5000, uid_source = 0xf76afeb4 "userdb lookup", gid_source = 0xf76afeb4 "userdb lookup", home = 0xf813ea71 "/srv/vmail/xtlv.de/1000", chroot = 0xf8130a20 ""} error = 0xf75b9934 "4\210\024" #6 mail_storage_service_next (ctx=0xf814d118, user=0xf813da90, mail_user_r=0xffa3b304) at mail-storage-service.c:1444 No locals. #7 0xf75fd0ff in mail_storage_service_lookup_next (ctx=0xf81399b0, input=0xffa3b368, user_r=0xffa3b300, mail_user_r=0xffa3b304, error_r=0xffa3b360) at mail-storage-service.c:1477 user = 0xf813da90 ret = <optimized out> #8 0xf77832c9 in client_create_from_input (input=0xffa3b368, fd_in=15, fd_out=15, client_r=0xffa3b35c, error_r=0xffa3b360) at main.c:228 user = 0x81a4 mail_user = 0xc34a5 ns = 0xf814d000 client = 0xffa3b304 imap_set = 0xffa3b360 lda_set = 0xffa3b304 errstr = 0xf814db90 "" mail_error = 49663 #9 0xf77834ea in login_client_connected (login_client=0xf813b450, username=0xf81300c8 "1000(a)xtlv.de", extra_fields=0xf81300ac) at main.c:316 input = {module = 0xf778b616 "imap", service = 0xf778b616 "imap", username = 0xf81300c8 "1000(a)xtlv.de", session_id = 0xf813b4c0 "doleKgxJ3s8l6zee", session_id_prefix = 0x0, session_create_time = 0, local_ip = { family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "T&K\217", '\000' <repeats 11 times>, __u6_addr16 = {9812, 36683, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {2404066900, 0, 0, 0}}}, ip4 = {s_addr = 2404066900}}}, remote_ip = { family = 2, u = {ip6 = {__in6_u = {__u6_addr8 = "%\353\067\236", '\000' <repeats 11 times>, __u6_addr16 = {60197, 40503, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {2654464805, 0, 0, 0}}}, ip4 = {s_addr = 2654464805}}}, local_port = 0, remote_port = 0, userdb_fields = 0xf81300ac, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0, debug = 0} client = 0xf750331c <t_malloc0+44> flags = <optimized out> error = 0xf8130130 "auth_token=3763fd48bfdfeea2a3617cbda148915c19e125fa" __FUNCTION__ = "login_client_connected" #10 0xf749b0d1 in master_login_auth_finish (client=0xf813b450, auth_args=0xf814d000, auth_args@entry=0xf81300a8) at master-login.c:210 login = 0xf813ab88 service = 0xf81383e8 __FUNCTION__ = "master_login_auth_finish" #11 0xf749b6a9 in master_login_postlogin_input (pl=0xf813d720) at master-login.c:284 login = 0xf813ab88 buf = "1000@xtlv.de\tquota_rule=*:storage=5M\tuid=5000\tgid=5000\thome=/srv/vmail/xtlv.de/1000\tauth_token=3763fd48bfdfeea2a3617cbda148915c19e125fa\nQ\345td", '\000' <repeats 20 times>, "\006\000\000\000\020\000\000\000R\345td\260.\000\000\260>\000\000\260>\000\000P\001\000\000P\001\000\000\004\000\000\000\001\000\000\000"... auth_args = 0xf81300a8 p = 0xf81300c0 ret = -1 fd = -1 ---Type <return> to continue, or q <return> to quit--- #12 0xf751b73e in io_loop_call_io (io=0xf813b358) at ioloop.c:599 ioloop = 0xf81384b8 t_id = 2 __FUNCTION__ = "io_loop_call_io" #13 0xf751cece in io_loop_handler_run_internal (ioloop=0xf81384b8) at ioloop-epoll.c:223 ctx = 0xf813a210 list = 0xf813b390 io = <optimized out> tv = {tv_sec = 59, tv_usec = 999219} events_count = <optimized out> msecs = <optimized out> ret = 1 i = <optimized out> j = 0 call = <optimized out> __FUNCTION__ = "io_loop_handler_run_internal" #14 0xf751b7ef in io_loop_handler_run (ioloop=0xf81384b8) at ioloop.c:648 No locals. #15 0xf751b999 in io_loop_run (ioloop=0xf81384b8) at ioloop.c:623 __FUNCTION__ = "io_loop_run" #16 0xf749d985 in master_service_run (service=0xf81383e8, callback=0xf7782b60 <client_connected>) at master-service.c:641 No locals. #17 0xf7766ece in main (argc=2, argv=0xffa3bab4) at main.c:460 set_roots = {0xf779d2a0 <imap_setting_parser_info>, 0xf7732ac0 <lda_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0xf8130030 "ta_rule=*:storage=5M", postlogin_socket_path = 0xf8130060 "/vmail/xtlv.de/1000", postlogin_timeout_secs = 60, callback = 0xf7783430 <login_client_connected>, failure_callback = 0xf7782cc0 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = <optimized out> auth_socket_path = <optimized out> c = <optimized out> doveconf -n # 2.2.devel (b3443fc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (0d78a30) # OS: Linux 3.2.64.stk32 x86_64 Debian 9.0 auth_debug = yes auth_debug_passwords = yes auth_failure_delay = 5 secs auth_master_user_separator = * auth_mechanisms = digest-md5 cram-md5 auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot/dov_debug.log deliver_log_format = msgid=%m: %$: DeliveryTime=%{delivery_time}: LMTPSessionTime=%{session_time} dict { expire = db:/var/lib/dovecot/expire.db } doveadm_password = # hidden, use -P to show it doveadm_port = 1090 imap_logout_format = in=%i out=%o MailHeaderR=%{fetch_hdr_count} MailBodyR=%{fetch_body_count} MailDel=%{deleted} MailExpung=%{expunged} MailTrash=%{trashed} lmtp_rcpt_check_quota = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_debug = yes mail_gid = vmail mail_location = sdbox:%h/sdbox mail_plugins = " quota mail_log notify expire zlib acl notify replication" mail_privileged_group = vmail mail_uid = vmail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { ignore_on_failure = no inbox = no list = children location = sdbox:%%h/sdbox prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = scheme=PLAIN username_format=%Lu /etc/dovecot/user_pw/passwd driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/srv/vmail/shared-db/shared-mailboxes expire = Trash 2h expire_cache = yes expire_dict = proxy::expire mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename save mailbox_create mail_log_fields = uid box msgid size subject mail_replica = tcps:mx00.vtlx.cn quota = dict:User quota::file:%h/dovecot-quota quota_exceeded_message = Die Mailbox des Empfaengers ist voll -- Quota exceeded -- Please contact <postmaster(a)xtlv.cn> quota_grace = 12%% quota_rule = *:storage=350M quota_rule2 = Trash:storage=+15%% quota_status_nouser = DUNNO quota_status_overquota = "552 5.5.2 Die Mailbox des Empfaengers ist voll ## Mailbox is full ## Please contact <postmaster(a)xtlv.cn>" quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=80%% quota-warning 80 %u quota_warning4 = storage=70%% quota-warning 70 %u sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz zlib_save_level = 6 } pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m/%{deleted_bytes}, size=%s postmaster_address = postmaster(a)xtlv.cn protocols = " imap lmtp sieve pop3" service aggregator { fifo_listener replication-notify-fifo { mode = 0660 user = vmail } unix_listener replication-notify { mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service config { unix_listener config { mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service doveadm { inet_listener { address = 84.38.75.143, 2a00:5080:1:16::8 port = 1090 ssl = yes } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = 84.38.75.143 port = 993 ssl = yes } } service imap-postlogin { executable = script-login -d /usr/local/sbin/dov-last-login-imap.sh user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { address = 84.38.75.143 port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = 84.38.75.143 port = 995 ssl = yes } } service pop3-postlogin { executable = script-login -d /usr/local/sbin/dov-last-login-pop3.sh user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl = required ssl_ca = </etc/dovecot/certs/sub.class2.server.ca.pem ssl_cert = </etc/dovecot/keys/vtlx.cn-wc-2015.crt ssl_cipher_list = ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA:!RC4:!aNULL:!eNULL:!LOW:!EXPORT:!DES:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!NULL:!DH:!EDH:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_client_ca_file = /etc/dovecot/certs/ca-bundle-2016.pem ssl_dh_parameters_length = 2048 ssl_key = # hidden, use -P to show it ssl_parameters_regenerate = 1 weeks ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 ssl_verify_client_cert = yes userdb { args = username_format=%Lu /etc/dovecot/user_pw/passwd driver = passwd-file } protocol lmtp { mail_plugins = " quota mail_log notify expire zlib acl notify replication quota sieve" } protocol imap { mail_max_userip_connections = 12 mail_plugins = " quota mail_log notify expire zlib acl notify replication imap_quota imap_zlib imap_acl" } protocol pop3 { mail_plugins = " quota mail_log notify expire zlib acl notify replication quota" } -- Persönlich IS0-Zertifiziert in angewandter Kompetenzsimulation.

2 1

Could not login as root or other Linux user account
by Basdove 21 Feb '17

21 Feb '17

Ubuntu server 16.04.2samba has upgraded from As per repository (latest version)I was configuring samba as per document from wiki "ActivedirectoryWINbindHowto"After editing the common-account and common-auth I rebooted the server.I could notlogin as root or any Linux user. Server says "Incorrect login" But I tried with all otherLinux user login which  are all logged well before inducing  root.How to login nowBelow is part of document :------------------------------------------------------------------------------------------------------------------------------- Note: You can use pam-auth-update to add the necessary entries for winbind authentication. If you installed libpam-winbind above, this step is all you need to do to configure pam. You may want to add the line to automatically create the home directory.sudo pam-auth-updateThis PAM configuration does not acquire a Kerberos TGT at login. To acquire a ticket, use kinit after logging in, and consider using kdestroy in a logout script.file: /etc/pam.d/common-accountaccount sufficient       pam_winbind.soaccount required         pam_unix.sofile: /etc/pam.d/common-authauth sufficient pam_winbind.soauth sufficient pam_unix.so nullok_secure use_first_passauth required   pam_deny.so-------------------------------------------------------------------------------------------------------------------- 

2 1

doveadm: Fatal: All your namespaces have a location setting
by Ben 21 Feb '17

21 Feb '17

Hi, I am trying to migrate mail from an old server and am receiving the following error : doveadm(user(a)example.com): Fatal: All your namespaces have a location setting. Only namespaces with empty location settings are converted. (One namespace should default to mail_location setting) I found an old thread (http://www.dovecot.org/list/dovecot/2012-September/068269.html) that referred to "location" being set. However this is not the case with me, my config reads: namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } My dsync conf looks as follows : imapc_host = old-server.example.com imapc_user = %u imapc_features = rfc822.size imapc_features = $imapc_features fetch-headers mail_prefetch_count = 20 imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = yes The command I'm calling is: sudo -u my_dovecot_user doveadm -c /etc/dovecot/dsync_config.conf -o mail_fsync=never backup -R -u user(a)example.com imapc:

3 3

Re: How to dsync mdbox compressed to maildir uncompressed
by Daniel Betz 21 Feb '17

21 Feb '17

Hi Timo, thank you for the hint, but it doesnt seems to work. doveadm sync -u imap(a)test.de 'doveadm -o mail="maildir:~/Maildir" -o "mail_plugins=quota" dsync-server -u imap(a)test.de' Also tried -o mail=~/Maildir .. -o maildir:~/Maildir .. The log throws an error: Feb 21 13:05:35 doveadm: Error: Panic: io_add(0x1) called twice fd=9, callback=0x7f49baa06840 -> 0x7f49ba991e30 Feb 21 13:05:35 doveadm: Error: Error: Raw backtrace: /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(+0x92d70) [0x7f49ba9efd70] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f49ba9efdda] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f49ba98b4e0] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(ioloop_iolist_add+0x83) [0x7f49baa03dc3] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f49baa046db] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(+0xa599f) [0x7f49baa0299f] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(io_add+0xd) [0x7f49baa02a4d] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(master_service_io_listeners_add+0x65) [0x7f49ba9916d5] -> /usr/local/dovecot2/lib/dovecot/libdovecot.so.0(master_service_init_finish+0xb7) [0x7f49ba9917a7] -> /usr/local/dovecot2/bin/doveadm(main+0x189) [0x4143a9] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f49ba5bcb15] -> /usr/local/dovecot2/bin/doveadm() [0x414785] Feb 21 13:05:35 dsync-local(imap(a)test.de): Error: read(remote) failed: EOF (version not received) Feb 21 13:05:35 dsync-local(imap(a)test.de): Error: Remote command died with signal 6: doveadm -o mail="maildir:~/Maildir" -o "mail_plugins=quota" dsync-server -u imap(a)test.de dsync-server Regards, Daniel Von: Timo Sirainen [mailto:tss@iki.fi] Gesendet: Dienstag, 21. Februar 2017 12:01 An: Daniel Betz <dbetz(a)df.eu> Cc: dovecot(a)dovecot.org Betreff: Re: How to dsync mdbox compressed to maildir uncompressed On 21 Feb 2017, at 12.49, Daniel Betz <dbetz(a)df.eu> wrote: Hello, we are using doveadm sync to export mdbox to maildir format, so we can use an external tool to convert into an pst file. Since we have enabled zlib compression doveadm sync always exports the maildir gzip compressed. Are there any ways to prevent the doveadm sync to export the maildir compressed ? Have tried this: doveadm -o "maildir_copy_with_hardlinks=no" sync -u imap(a)test.de maildir:~/Maildir >From Wiki: If you want to use dsync to convert to a compressed Maildir you may need -o maildir_copy_with_hardlinks=no (this is set to yes by default and will prevent compression). Run it via two processes so you can give separate settings for them, something like: doveadm sync -u imap(a)test.d 'doveadm -o mail=~/Maildir -o mail_plugins=everything-but-zlib dsync-server'

1 0

How to dsync mdbox compressed to maildir uncompressed
by Daniel Betz 21 Feb '17

21 Feb '17

Hello, we are using doveadm sync to export mdbox to maildir format, so we can use an external tool to convert into an pst file. Since we have enabled zlib compression doveadm sync always exports the maildir gzip compressed. Are there any ways to prevent the doveadm sync to export the maildir compressed ? Have tried this: doveadm -o "maildir_copy_with_hardlinks=no" sync -u imap(a)test.de maildir:~/Maildir >From Wiki: If you want to use dsync to convert to a compressed Maildir you may need -o maildir_copy_with_hardlinks=no (this is set to yes by default and will prevent compression). Regards, Daniel # 2.2.27 (c0f36b0): /usr/local/dovecot2/etc/dovecot/dovecot.conf doveconf: Warning: service auth { client_limit=60000 } is lower than required under max. load (500500) # OS: Linux 3.10.0-327.36.3.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_cache_negative_ttl = 1 mins auth_cache_size = 64 M auth_cache_ttl = 2 hours auth_mechanisms = plain login auth_username_chars = base_dir = /var/run/dovecot/ debug_log_path = /dev/null default_login_user = dovecot default_vsz_limit = 750 M disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 first_valid_gid = 1001 first_valid_uid = 1001 info_log_path = /var/log/dovecot/messages lda_mailbox_autocreate = yes lda_original_recipient_header = X-Envelope-To log_path = /dev/stderr login_log_format_elements = user=[%u] method=%m rip=%r lip=%l %c mail_gid = 1001 mail_location = mdbox:~:INDEX=%h/INDEX mail_plugins = quota notify mail_log zlib mail_uid = 1001 mbox_write_locks = fcntl namespace { inbox = yes location = mailbox Drafts { auto = no special_use = \Drafts } mailbox "Gesendete Elemente" { auto = no special_use = \Sent } mailbox "Infizierte Objekte" { auto = no special_use = \Junk } mailbox Sent { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = no special_use = \Trash } prefix = separator = . type = private } namespace inbox { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap2.conf driver = ldap } plugin { quota = dict:User quota::file:%h/mdbox/dovecot-quota quota_rule1 = Trash:storage=+100M quota_rule2 = INBOX.Trash:storage=+100M quota_warning = storage=85%% quota-warning 85 %u quota_warning1 = storage=95%% quota-warning 95 %u quota_warning2 = storage=99%% quota-warning 99 %u zlib_save = gz zlib_save_level = 6 } replication_max_conns = 30 sendmail_path = /usr/local/exim/bin/exim service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = popuser } unix_listener replication-notify { mode = 0666 user = popuser } } service anvil { client_limit = 60000 } service auth { client_limit = 60000 unix_listener auth-userdb { mode = 0666 user = popuser } unix_listener auth { mode = 0666 user = popuser } } service config { unix_listener config { user = popuser } } service dict { unix_listener dict { mode = 0666 user = popuser } } service dns_client { process_limit = 6000 process_min_avail = 12 unix_listener dns-client { mode = 0666 user = popuser } } service doveadm { inet_listener { port = 12345 } user = popuser } service imap-login { chroot = login client_limit = 6000 process_limit = 100 process_min_avail = 16 service_count = 0 } service imap { executable = /usr/local/dovecot2/libexec/dovecot/imap process_limit = 250000 process_min_avail = 50 service_count = 250 } service ipc { client_limit = 60000 unix_listener ipc { mode = 0650 user = dovecot } unix_listener login/ipc-proxy { mode = 0650 user = dovecot } } service lmtp { unix_listener lmtp { mode = 0666 user = popuser } } service pop3-login { chroot = login client_limit = 6000 process_limit = 100 process_min_avail = 16 service_count = 0 } service pop3 { executable = /usr/local/dovecot2/libexec/dovecot/pop3 process_limit = 250000 process_min_avail = 50 service_count = 250 } service quota-warning { executable = script /usr/local/dovecot2/bin/quota-warning.sh unix_listener quota-warning { mode = 0600 user = popuser } user = popuser } service replicator { unix_listener replicator-doveadm { mode = 0600 user = popuser } } ssl_cert = </usr/local/dovecot2/etc/dovecot/certs/cert1.crt ssl_dh_parameters_length = 2048 ssl_key = # hidden, use -P to show it ssl_protocols = TLSv1.2 TLSv1.1 TLSv1 !SSLv3 !SSLv2 userdb { driver = prefetch } userdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol doveadm { mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ passdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-doveadm.conf driver = ldap name = } userdb { args = /usr/local/dovecot2/etc/dovecot/dovecot-doveadm.conf driver = ldap name = } } protocol lda { info_log_path = /var/log/dovecot-lda log_path = /var/log/dovecot-lda log_timestamp = "%b %d %H:%M:%S " mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ postmaster_address = %u } protocol imap { imap_client_workarounds = mail_max_userip_connections = 100 mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ mail_plugins = quota notify mail_log zlib imap_quota imap_zlib } protocol pop3 { mail_plugin_dir = /usr/local/dovecot2/lib/dovecot/ pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = yes pop3_fast_size_lookups = yes pop3_reuse_xuidl = no pop3_uidl_format = %g }

3 2

IMAP directory structure.
by C5ace 21 Feb '17

21 Feb '17

Hello, I use Dovecot 2.1.7. and like to know how to force the IMAP directory structure to be: The IMAP server's desired directory structure is: root@server-2:/var/vmail/c5ace.com/test/Maildir# /.INBOX /.INBOX.Archives /.INBOX.Drafts /.INBOX.Junk /.INBOX.Sent /.INBOX.Templates /.INBOX.Trash and to prevent the mail clients like Thunderbird, Claws Mail, etc. to add additional out of tree directories. Thanks in advance. Elmar

2 1

v2.2.28 release candidate released
by Timo Sirainen 21 Feb '17

21 Feb '17

http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.28.rc1.tar.gz.sig Pretty large release. Please test before the final v2.2.28, which should be out in a few days. BTW. Our plan is to start making new releases approximately every month from now on. * director: "doveadm director move" to same host now refreshes user's timeout. This allows keeping user constantly in the same backend by just periodically moving the user there. * When new mailbox is created, use initially INBOX's dovecot.index.cache caching decisions. * Expunging mails writes GUID to dovecot.index.log now only if the GUID is quickly available from index/cache. * imap: When BINARY FETCH sees invalid content, return NO [PARSE] reply instead of [UNKNOWNCTE] (which is now used only for actually unknown Content-Transfer-Encoding headers). * pop3c: Increase timeout for PASS command to 5 minutes. * Mail access errors are no longer ignored when searching or sorting. With IMAP the untagged SEARCH/SORT reply is still sent the same as before, but NO reply is returned instead of OK. + Make dovecot.list.index's filename configurable. This is needed when there are multiple namespaces pointing to the same mail root (e.g. lazy_expunge namespace for mdbox). + Add size.virtual to dovecot.index when folder vsizes are accessed (e.g. quota=count). This is mainly a workaround to avoid slow quota recalculation performance when message sizes get lost from dovecot.index.cache due to corruption or some other reason. + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them in lib-dsasl for client side. + auth: Support filtering by SASL mechanism: passdb { mechanisms } + Shrink the mail processes' memory usage by not storing settings duplicated unnecessarily many times. + imap: Add imap_fetch_failure setting to control what happens when FETCH fails for some mails (see example-config). + imap: Include info about last command in disconnection log line. + imap: Created new SEARCH=X-MIMEPART extension. It's currently not advertised by default, since it's not fully implemented. + fts-solr: Add support for basic authentication. + Cassandra: Support automatically retrying failed queries if execution_retry_interval and execution_retry_times are set. + doveadm: Added "mailbox path" command. + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the wanted fields only if it doesn't require opening the email. + mail_vsize_bg_after_count setting added (see example-config). + mail_sort_max_read_count setting added (see example-config). - Index files: day_first_uid wasn't updated correctly since v2.2.26. This caused dovecot.index.cache to be non-optimal. - imap: SEARCH/SORT may have assert-crashed in client_check_command_hangs - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. - imap: Running time in tagged command reply was often wrongly 0. - search: Using NOT n:* or NOT UID n:* wasn't handled correctly - director: doveadm director kick was broken - director: Fix crash when using director_flush_socket - director: Fix some bugs when moving users between backends - imapc: Various error handling fixes and improvements - master: doveadm process status output had a lot of duplicates. - autoexpunge: If mailbox's rename timestamp is newer than mail's save-timestamp, use it instead. This is useful when autoexpunging e.g. Trash/* and an entire mailbox is deleted by renaming it under Trash to prevent it from being autoexpunged too early. - autoexpunge: Multiple processes may have been trying to expunge the same mails simultaneously. This was problematic especially with lazy_expunge plugin. - auth: %{passdb:*} was empty in auth-worker processes - auth-policy: hashed_password was always sent empty. - dict-sql: Merge multiple UPDATEs to a single statement if possible. - fts-solr: Escape {} chars when sending queries - fts: fts_autoexpunge_exclude = \Special-use caused crashes - doveadm-server: Fix leaks and other problems when process is reused for multiple requests (service_count != 1) - sdbox: Fix assert-crash on mailbox create race - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve was used. especially %{storage_id} was broken.

8 10

Released Pigeonhole v0.4.17.rc1 for Dovecot v2.2.28.rc1.
by Stephan Bosch 21 Feb '17

21 Feb '17

Hello Dovecot users, Several bugs were found in the course of the last few months, but no new features were created. So, this will be strictly a bug-fixing release. The previous Pigeonhole release should still work with the new Dovecot release, which means you are not required to upgrade. However, my advice would be to upgrade once this release candidate matures into a final release, since at least the first bug listed below is pretty severe. Changelog v0.4.17: - LDA Sieve plugin: Fixed handling of an early explicit keep during multiscript execution. Action side-effects and the message snapshot would be lost at the final stage where the implicit keep is evaluated. This could result in the IMAP flags assigned to the message to be forgotten or that headers modified by the "editheader" extension would revert to their original state. - file script storage: Amended the up-to-date time stamp comparison for on-disk binaries to include nanoseconds. This will fix problems occurring when both binary and script are saved within the same second. This fix is ineffective on older systems that have no support for nanoseconds in stat() timestamps, which should be pretty rare nowadays. - file script storage: Improve saving and listing permission error to include more details. - imapsieve plugin: Make sure "INBOX" is upper case in static mailbox rules. Otherwise, the mailbox name would never match, since matching is performed case-sensitively and Dovecot only returns the upper-cased "INBOX". - imapsieve plugin: Fixed assert failure occurring when used with virtual mailboxes. - doveadm sieve plugin: Fixed crash when setting Sieve script via attribute's string value. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.17… http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.17… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

3 2

Sieve not filtering
by Ben 20 Feb '17

20 Feb '17

Hi, I have copied accross a known-good sieve file from a working server and its not filtering. Everything just gets chucked into INBOX. doveconf-n at the bottom of this mail Feb 17 16:05:20 server postfix/smtpd[51562]: 7FA5E12CBBC: client=unknown[192.168.167.57] Feb 17 16:05:23 server postfix/cleanup[51565]: 7FA5E12CBBC: message-id=<> Feb 17 16:05:23 server postfix/qmgr[45471]: 7FA5E12CBBC: from=<sender(a)example.com>, size=182, nrcpt=1 (queue active) Feb 17 16:05:23 server dovecot: lmtp(51467): Connect from local Feb 17 16:05:23 server dovecot: auth-worker(51568): passwd(recipient(a)example.com): unknown user Feb 17 16:05:23 server dovecot: lmtp(51467, recipient(a)example.com) 1JK2B0Mfp1gLyQAAHLpRfg: sieve: msgid=unspecified: stored mail into mailbox 'INBOX' Feb 17 16:05:23 server dovecot: lmtp(51467): Disconnect from local: Successful quit Feb 17 16:05:23 server postfix/lmtp[51566]: 7FA5E12CBBC: to=<recipient+something(a)example.com>, orig_to=<recipient+something(a)example.com>, relay=server.example.com[private/dovecot-lmtp], delay=4.9, delays=4.9/0.01/0/0.05, dsn=2.0.0, status=sent (250 2.0.0 <bp+nanog(a)example.com> 1JK2B0Mfp1gLyQAAHLpRfg Saved) Feb 17 16:05:23 server postfix/qmgr[45471]: 7FA5E12CBBC: removed This is the syntax I'm using in my sieve file: require ["fileinto","envelope"]; if anyof (address :is :all "to" ["recipient+something@example.com”,”mailing_list@example.com”], header :contains ["Cc", "Delivered-To"] ["recipient+something@example.com”,”mailing_list@example.com”]) { fileinto “THIS_FOLDER”; stop; } # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.6.1.el7.x86_64 x86_64 CentOS Linux release 7.3.1611 (Core) auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1 first_valid_uid = 1000 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body environment mailbox date ihave enotify mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = ~/.dovecot.sieve } protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = its_virtmail mode = 0660 user = its_virtmail } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = its_virtmail } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieves { address = port = 5190 ssl = yes } } ssl = required ssl_cert = <REMOVED ssl_cipher_list = REMOVED ssl_dh_parameters_length = 2048 ssl_key = </REMOVED ssl_prefer_server_ciphers = yes ssl_protocols = !SSlv2 !SSLv3 userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = sieve postmaster_address = me(a)example.com } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = postmaster quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 20 }

5 5

Issue connecting to dovecot from remote machine
by Antoine Sirven 20 Feb '17

20 Feb '17

Hi, I've set up a postfix +dovecot configuration on my debian jessie. But I have a connection issue. When I try to connect from thunderbird it doesn not work. When I check out my debug logs I get : auth-worker(22252): Info: pam(myuser,hostIP): pam_authenticate() failed: Authentication failure (password mismatch?) (given password: correctPassword) Running doveadm auth test tells me I can authenticate with the same password. So I tried connecting via openssl. When I connect from my local host, everything goes fine : * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. a login myUser myPassword a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in But when I run the same thing from my remote laptop, I never get any reply from the imap server. It's as if the request wasn't reaching the server. Although I do get a first reply from the server, I just can't log in. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot ready. a login myUser myPassword (... and then nothing...) The port is properly open when I run nmap. I really don't get what's wrong. Any of you has any idea? Here are my versions & conf : # dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab120.16 x86_64 Debian 8.7 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = yes disable_plaintext_auth = no info_log_path = /var/log/maildebug.log mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = failure_show_msg=yes session=yes dovecot driver = pam } passdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } ssl = required ssl_cert = </etc/ssl/certs/mailcert.pem ssl_key = </etc/ssl/private/mail.key userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } verbose_ssl = yes Thanks a lot

2 2

Re: Upgrade from 1.2 to 2.2
by Joseph Tam 16 Feb '17

16 Feb '17

Eduardo M KALINOWSKI writes in reesponse to Gandalf Corvotempesta post: >> I've already read that, and as wrote previously, everything broke down. >> dovecont -n wasn't able to convert the configuration file and dovecot >> wasn't started properly. >> The only way to fix was to downgrade. >> >> As this is a production server, I would like to avoid this kind of issue..... > > Set up a server replicating exactly the configuration you have at > present. Doesn't need to be anything fancy, a virtual machine is > enough. Actually, a virtual machine is a great tool if you can make > snapshots and then rollback to known states if any attempt at a change > fails. If you absolutely can't test on a VM or another test host, you can run the new version along side your current version in another installation directory. Configure your new installation to run on alternate ports. You can work off copies of mail data as well if you really want to isolate your test. Then you can gradually phase in your new version by swapping alternate port/data with production port/data. Joseph Tam <jtam.home(a)gmail.com>

1 0

doveadm backup and quota
by florent＠coppint.com 15 Feb '17

15 Feb '17

Hi, I'm looking to convert all mailboxes from Maildir format to mdbox. When I try my procedure on a user having ~45GB of messages, its quota is growing very fast and is now up to 66GB. I run this command : doveadm -v backup -u user(a)domain.com mdbox:/mnt/maildata1/domain.com/user/mdbox I expected this command to make a copy of user mailbox, it does not make sense to count the backup in its quota. Maybe quota is simply recalculated ? But running a "du -sh" on its home directory is showing a 46G disk usage ! (backup is not stored in home dir) Can someone tell me what happens on my setup ? I use Dovecot 2.2.24. Thank you. Florent

1 1

Upgrade from 1.2 to 2.2
by Gandalf Corvotempesta 15 Feb '17

15 Feb '17

Hi, I have a production server running Debian Squeeze with Dovecot 1.2 I would like to upgrade everything to Jessie, running 2.2 Last time I did something similiar, but from Lenny to Squeeze, the whole dovecot installation broke. Any suggestion on how to upgrade everything ? Can I test our current configuration with a newer dovecot version to be sure that everything would be converted properly with "doveconf -n -c /etc/dovecot/dovecot.conf > dovecot-2.conf" ? Thank you

4 6

Re: dovecot config for 1500 simultaneous connection
by Rajesh M 15 Feb '17

15 Feb '17

christian the servers i currently own are dell servers. The servers i plan to buy are Dell R530, 2U rack servers with 8 x 3.5 inch drives, with 64 gb ram each, Hardware raid. I am thinking of 2 X 300 gb ssds raid1 and 6 x 2 tb drives in raid10 for data. I do not have any experience in setting up drdb (that would be my next step) ... primarily using standalone servers with hardware level redundancy. For backup-ups i rsync data to backup servers, also use qmail taps for backing up sent/recd emails on a per user basis. i am using mysql authentication backend my mail server is qmailtoaster, ie with vpopmail, spamdyke, simscan. you had mentioned as follows ############## With 4000 users and 5 logins/s I can't imagine anything that wouldn't be able to: a) put all of the relevant data into RAM and b) thus handle auth requests in very timely manner. The example I gave earlier about my servers with 140 logins/s authenticates against LDAP and isn't very busy doing so while also fitting all 2.7GB of data for about 1 million users into RAM. ############## >>>>>>>>>>>>>>>>>>how do i put all relevant data into the RAM ? thanks rajesh ----- Original Message ----- From: Christian Balzer [mailto:chibi@gol.com] To: dovecot(a)dovecot.org Sent: Tue, 14 Feb 2017 16:40:01 +0900 Subject: Hello, On Mon, 13 Feb 2017 22:53:23 +0530 Rajesh M wrote: > thanks for your help > > happy to say that the performance dramatically improved after i use the high performance settings from here > http://wiki.dovecot.org/LoginProcess > That's why that page is there. > grep Login: /var/log/mail.log.1 |wc -l > with the mail.log being of a typical, busy day. > 412992 > So about 5 login/s, not that busy, but that of course also depends on your HW and authentication backend. > i also picked up the imap and pop3 connections during peak hours > [root@ns1 domains]# doveadm who | awk '/imap/{m+=$2}/pop3/{n+=$2}END{print m,n}' > username # proto (pids) (ips) > > max figures i got was 535 for imap and 97 for pop > > > Question 1 > i wish to improve the performance further by caching the logins. current the same is kept disable because when user's change passwords then they are not able to immediately login with the new password for some time. How to solve this issue. > There is no solution, caching auth data is by default fraught with that sort of peril. And how did you determine that caching auth data would actually improve things? What is your authentication backend? With 4000 users and 5 logins/s I can't imagine anything that wouldn't be able to: a) put all of the relevant data into RAM and b) thus handle auth requests in very timely manner. The example I gave earlier about my servers with 140 logins/s authenticates against LDAP and isn't very busy doing so while also fitting all 2.7GB of data for about 1 million users into RAM. > > > further to the above. > i am planning to put up a large mail server with one of the following options > Don't plan 1, always plan a pair and then use whatever you're comfortable with to turn them into a HA cluster, like Pacemaker with DRBD. > option 1 > OS and mailqueue : 2 X 600 gb 15k rpm raid 1 What kind of RAID, HW or MD? > data : 4 X 2000 gb raid 10 for data > backup : 2 X 2000 gb raid 10 for backup While with Linux MD a 2 disk RAID10 would be possible, it wouldn't be particular helpful (slower than a RAID 1 with writes). > 32 gb ram > > > option 2 > os and mail queue : 4 X 600 gb 15k rpm raid 10 Those 15k HDDs tend to be expensive. Estimate your expected writes per day (iostat on an existing server can give you some hints) and get 2 SSDs instead, DC S3520s if 1 DWPD is sufficient, DC S3610s if you need 3 DWPD (doubtful), S3710 if you need 10 DWPD (very doubtful). Similar for Samsung DC level SSDs, check their specs. For example, my busiest mailbox servers only write about 250KB/s averaged, which mean about 50GB/day and thus would be a fit even for a small S3520. I'm still using 200GB S3710s because I can afford them, like the speed and never ever want to worry about wearout. > data : 4 X 2000 gb raid 10 for data > 32 gb ram > > i am having the OS and mail queue on primary drives > > i understand that Raid10 give a faster write and read whereas Raid 1 will have slow writes and fast reads > While that is true, you're missing the real difference here, the number of HDDs. With 4 of them in the RAID10 they will be twice as fast as the 2 disk RAID1. This can be tweaked even further with the various RAID 10 layout options, "man md". > the question is will there be huge performance difference between Raid1 and Raid 10 when it comes small files like mail queue ? > Again as explained above, double the IOPS, so yes. Christian > > thanks > rajesh > ----- Original Message ----- > From: Christian Balzer [mailto:chibi@gol.com] > To: dovecot(a)dovecot.org > Cc: kevin(a)my.walr.us > Sent: Mon, 13 Feb 2017 10:46:13 +0900 > Subject: > > > Hello, > > On Sun, 12 Feb 2017 08:27:21 -0500 KT Walrus wrote: > > > Thanks for the info. I do have one further question for you. On your servers that are currently handling 50k IMAP sessions, how many users does that correspond to? Since many users will have multiple IMAP sessions on multiple devices, I’d like to hear about some real-world numbers that could be used for budgeting a new project like mine. > > > > Those servers would actually be the wrong ones to look at, as they are > primarily accessed by the aforementioned broken client, so the numbers are > somewhat skewed. > However looking at other servers with a more "normal" user spread things > aren't actually too different. > > The average number of sessions per user tends to be 2. > The clear majority (over 50%) only has one session open (people with well > behaved and configured clients watching the INBOX mostly). > Another 30% has 2 sessions open, again the typical state of this would be > clients watching another mailbox besides INBOX, typically SENT. > The rest has 3 and more sessions open. > > The number of sessions could of course be drastically reduced if any > client would support IMAP NOTIFY, alas none that I'm aware of do. > > Lastly no more than 60% of the session seem to be in IDLE at any given > time, so my comments about RAM and IMAP hibernation effectiveness stand. > > > Also, do you use Dovecot IMAP proxies in front of your backend servers? If so, how many IMAP sessions can one proxy server handle (assuming the proxy does authorization using MySQL running on a separate server)? And, could the proxy server be tuned to help in optimizing mostly IDLE backend sessions? > > > > Yes to Dovecot Proxying, of course. > > No idea about MySQL, with (Open)LDAP nothing is breaking a sweat at an > average of 140 logins per second (IMAP and POP) on the 2 proxy servers. > If you can fit your entire dataset into RAM it should be fine, my LDAP > servers fall into that category and take about 10% of a slow (1.2GHz, 34%, > power-save mode) core only to handle the that load (also 2 servers). > And the rate of logins/s is what you need to worry about most and optimize > for. > > The proxies will of course have to do the shuffling of data and SSL > en/de-coding, but again they're not particular busy with that. > > The number of sessions comes into play when looking at the number of login > processes on the proxies and their memory footprint. > An IMAP login process on the proxies in performance mode with a client > limit of 1000 will consume about 55MB at most. > So assume at least 55KB RAM per session. > > Read Timo's mail I linked to about IMAP hibernation, AFAIK nothing has > happened to make proxies more supportive for this though. > > Christian > > > On Feb 12, 2017, at 1:58 AM, Christian Balzer <chibi(a)gol.com> wrote: > > > > > > > > > Hello, > > > > > > On Fri, 10 Feb 2017 14:50:03 -0500 KT Walrus wrote: > > > > > >>> 1. 256GB of real RAM, swap is for chums. > > >> > > >> Are you sure that 100,000 IMAP sessions wouldn’t work well with SWAP, especially with fast SSD storage (which is a lot cheaper than RAM)? > > >> > > > > > > I'm sure about tax and death, not much else. > > > > > > But as a rule of thumb I'd avoid swapping out stuff on production servers, > > > even if it were to SSDs. > > > Incidentally the servers I'm talking about here have their OS and swap on > > > Intel DC S3710s (200GB) and the actual storage on plenty of 1.6TB DC > > > S3610s. > > > > > > Relying on the kernel to make swap decisions is likely to result in much > > > reduced performance even with fast SWAP when you're overcommitting things > > > on that scale. > > > > > > > > > But read on. > > > > > >> Seems that these IMAP processes are long lived processes (idling most of the time) that don’t need that much of the contents of real memory available for much of the life of the process. I use a database proxy in front of MySQL (for my web apps) so that there can be a large number of TCP connections to the proxy where the frontend requests are queued for execution using a small number of backend connections. > > >> > > >> Could Dovecot IMAP be re-written to be more efficient so it works more like MySQL (or other scalable data servers) that could handle a million or more IMAP sessions on a server with 32GBs or less of RAM? Those IMAP sessions aren’t doing much most of the time and shouldn’t really average 2MB of active data per session that needs to be resident in main memory at all times. > > >> > > > See IMAP hibernation: > > > https://www.mail-archive.com/dovecot@dovecot.org/msg63429.html <https://www.mail-archive.com/dovecot@dovecot.org/msg63429.html> > > > > > > I'm going to deploy/test this in production in about 2 months from now, > > > but if you look at the link and the consequent changelog entries you'll see > > > that it has certain shortcomings and bug fixes in pretty much each release > > > after it was introduced. > > > > > > But this is the correct way to tackle things, not SWAP. > > > > > > Alas I'm not expecting miracles and if more than 20% of the IMAP sessions > > > here will be hibernated at any given time I'd be pleasantly surprised. > > > > > > Because between: > > > > > > 1. Finding a sensible imap_hibernate_timeout. > > > > > > 2. Having well behaved clients that keep idling instead of restarting the > > > sequence (https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/ <https://joshdata.wordpress.com/2014/08/09/how-bad-is-imap-idle/>) > > > > > > 3. Having lots of mobile clients who either get disconnected (invisible to > > > Dovecot) or have aggressive IDLE timers to overcome carrier NAT timeouts > > > (a large mobile carrier here times out idle TCP sessions after 2 minutes, > > > forcing people to use 1 minute IDLE renewals, making 1. up there a > > > nightmare). > > > > > > 4. Having really broken clients (don't ask, I can't tell) which open IMAP > > > sessions, don't put them into IDLE and thus having them expire after 30 > > > minutes. > > > > > > the pool of eligible IDLE sessions isn't as big as it could be, in my case > > > at least. > > > > > >> My mail server isn’t that large yet as I haven’t fully deployed Dovecot outside my own small group yet, but it would be nice if scaling Dovecot IMAP to millions of users wasn’t limited to 50,000 IMAP sessions on a server... > > >> > > > > > > Scaling up is nice and desirable from a cost (rack space, HW) perspective, > > > but the scalability of things OTHER than Dovecot as I pointed out plus > > > that little detail of failure domains (do you really want half of your > > > eggs in one basket?) argue for scaling out after a certain density. > > > > > > I'm feeling my way there at this time, but expect more than 100k sessions > > > per server to be tricky. > > > > > > Lastly, when I asked about 500k sessions per server here not so long ago, > > > ( http://www.dovecot.org/list/dovecot/2016-November/106284.html <http://www.dovecot.org/list/dovecot/2016-November/106284.html> ) > > > Timo mentioned that he's not aware of anybody doing more than 50k per > > > server, something I got licked already and definitely will go to 100k > > > eventually. > > > > > > Regards, > > > > > > Christian > > >>> On Feb 10, 2017, at 11:07 AM, Christian Balzer <chibi(a)gol.com> wrote: > > >>> > > >>> On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote: > > >>> > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > >>>> > > >>>> Are you saying that Dovecot needs 2MB of physical memory per IMAP session? > > >>>> > > >>> That depends on the IMAP session, read the mailbox size and index size, > > >>> etc. > > >>> Some are significantly larger: > > >>> --- > > >>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > >>> 1033864 mail 20 0 97600 67m 54m S 0 0.1 0:01.15 imap > > >>> --- > > >>> > > >>> But yes, as somebody who has mailbox servers with 55k+ session the average > > >>> is around 1.6MB. > > >>> > > >>>> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP? > > >>>> > > >>> You will want: > > >> > > >>> 2. Understanding how to tune Dovecot and more importantly the overall > > >>> system to such a task (see that PID up there?). > > >>> 3. Be willing to deal with stuff like top and ps taking ages to start/run > > >>> and others like atop actually killing dovecot (performance wise, not > > >>> literally) when doing their obviously flawed cleanup on exit. Some things > > >>> clearly do NOT scale well. > > >>> > > >>> My current goal is to have 100k capable servers that work well, 200k in a > > >>> failover scenario, but that won't be particular enjoyable. > > >>> > > >>> Christian > > >>> > > >>>>> On Feb 10, 2017, at 3:58 AM, Christian Balzer <chibi(a)gol.com> wrote: > > >>>>> > > >>>>> On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote: > > >>>>> > > >>>>>> hello > > >>>>>> > > >>>>>> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously. > > >>>>>> > > >>>>> > > >>>>> Be very precise here, you expect to see 1500 as the result of > > >>>>> "doveadm who |grep pop3 |wc -l"? > > >>>>> > > >>>>> Because that implies an ungodly number of POP3 connects per second, given > > >>>>> the typically short duration of these. > > >>>>> > > >>>>> 1500 IMAP connections (note that frequently a client will have more than > > >>>>> the INBOX open and thus have more than one session and thus process on the > > >>>>> server) are a much easier proposition, provided they are of the typical > > >>>>> long lasting type. > > >>>>> > > >>>>> So can you put a number to your expected logins per second (both protocols)? > > >>>>> > > >>>>>> my server > > >>>>>> > > >>>>>> server configuration > > >>>>>> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 > > >>>>>> gb hdd for data (No raid) > > >>>>>> > > >>>>> No RAID and no other replication like DRBD? > > >>>>> Why would you even bother? > > >>>>> > > >>>>> How many users/mailboxes in total with what quota? > > >>>>> > > >>>>> 1500 IMAP sessions will eat up about 3GB alone. > > >>>>> You will want more memory, simply to keep all relevant SLAB bits (inodes, > > >>>>> dentries) in RAM. > > >>>>> > > >>>>> If you really have several hundreds logins/s, you're facing several > > >>>>> bottlenecks: > > >>>>> 1. Login processes themselves (easily fixed by high performance mode) > > >>>>> 2. Auth processes (that will depend on your backends, method mostly) > > >>>>> 3. Dovecot master process (spawning mail processes) > > >>>>> > > >>>>> The later is a single-threaded process, so it will benefit from a faster > > >>>>> CPU core. > > >>>>> It can be dramatically improved by enabling process re-usage, see: > > >>>>> http://wiki.dovecot.org/PerformanceTuning > > >>>>> > > >>>>> However that also means more memory usage. > > >>>>> > > >>>>> > > >>>>> > > >>>>> Christian > > >>>>> > > >>>>>> > > >>>>>> thanks > > >>>>>> rajesh > > >>>>>> > > >>>>> > > >>>>> [snip] > > >>>>> -- > > >>>>> Christian Balzer Network/Systems Engineer > > >>>>> chibi(a)gol.com Global OnLine Japan/Rakuten Communications > > >>>>> http://www.gol.com/ > > >>>> > > >>> > > >>> > > >>> -- > > >>> Christian Balzer Network/Systems Engineer > > >>> chibi(a)gol.com <mailto:chibi@gol.com> <mailto:chibi@gol.com <mailto:chibi@gol.com>> Global OnLine Japan/Rakuten Communications > > >>> http://www.gol.com/ <http://www.gol.com/> <http://www.gol.com/ <http://www.gol.com/>> > > > > > > > > > -- > > > Christian Balzer Network/Systems Engineer > > > chibi(a)gol.com <mailto:chibi@gol.com> Global OnLine Japan/Rakuten Communications > > > http://www.gol.com/ <http://www.gol.com/> > > > > -- Christian Balzer Network/Systems Engineer chibi(a)gol.com Global OnLine Japan/Rakuten Communications http://www.gol.com/

2 1

Re: dovecot config for 1500 simultaneous connection
by Joseph Tam 15 Feb '17

15 Feb '17

Rajesh M wrote: > i wish to improve the performance further by caching the logins. > current the same is kept disable because when user's change passwords > then they are not able to immediately login with the new password for > some time. How to solve this issue. Dovecot shouldn't be doing that. According to https://wiki2.dovecot.org/Authentication/Caching Data is used from the cache if it's not expired (auth_cache_ttl setting) - If authentication fails this time, but it didn't fail last time, it's assumed that the password has changed and a database lookup is done. As I read it, an authentication failure will cause an update of cache credentials. The problem I encountered is the converse -- a user changes their password, but forgets to update their mail reader's or web browser's stored password, which continues to work until the TTL expires. Another related security situation I've encountered is when a fraudster has phished a user's password. A user/admin changes the password, but forgets to invalidate dovecot's cached entry, allowing the fraudster contunuing access to the mail account until the TTL expires or user logs in with new credentials. I've been burnt by this one. Joseph Tam <jtam.home(a)gmail.com>

2 2