- dovecot - dovecot.org

Reporting on CephFS being ready to use with Dovecot
by Daniel van Ham Colchete 18 Aug '16

18 Aug '16

I would like to report that from version 10.2.3 on (next release), Ceph FS is working really well with Dovecot systems. For those that don't know, Ceph is a "distributed object store and file system designed to provide excellent performance, reliability and scalability.". We have used it here since 2013 very successfully, but never with our Dovecot setup. For more information go on http://ceph.com/. Since Ceph Jewel (the current version), Ceph FS is considered production ready by their team. With Ceph FS you have a cache-coherent POSIX-compliant [1] clustered file system, without most of the NFS shortcomings. Ceph have very nice features like online upgrades, online maintenance, constant deep scrubbing of replicated data, cache tiering (HD -> SSD -> etc), erasure coding (clustered RAID6 for really old email - I'm not using), etc. Ceph is very complex to operate but very flexible and robust. This year we moved our Dovecot servers to a Ceph based system, found one bug there (http://tracker.ceph.com/issues/15920) when Dovecot's LMTP was delivery an email, and the fix is about to be released on version 10.2.3. I have been using a fix-build here for a couple of months without issue. So, now I'm glad to share with you guys that it works really well! My setup involves two clusters, each with about 30k-40k users. Each cluster will have two HD storages (with 6TB HDs), two SSD storages (with Intel 480GB SSDs) and two frontends. In a few months will add a third server of each type. Clusters work better with 3's. Here we used mdbox as it is better performant on Ceph for maintenance tasks, since each file is an object on Ceph and maintenance costs increase with the number of objects. We created two base directories: - /srv/dovecot/mail/%d/%n - stored on HDs with the most recent files cached on SSDs, thanks to Ceph Cache Tiering. Also, the directory structure itself is stored on SSDs, so dir listings are very fast (Ceph FS Metadata). - /srv/dovecot/index/%d/%n - stored only on SSDs, thanks for Ceph FS file layout. On our setup about 17% of the IOPs are going to HDs, the rest will go to SSDs, even though SSDs are less than 5% of the space. This is a matter of tuning the cache tiering parameters, but we didn't look at that yet. That setup is working like a charm, performance is about 53% better than when we were using NFS on the same hardware. Our previous DRBD+Heartbeat setup didn't allow for online maintenance and had a few problems. Now we can do 100% online maintenance on storage without users noticing, and on frontends with just a reconnect but without any downtime. Ceph is hard to learn at first but those with bigger setups and stronger SLAs will want to take a look at that. I really recommend that the Dovecot community take at look at that setup. Good luck! Best, Daniel Colchete [1] http://docs.ceph.com/docs/hammer/dev/differences-from-posix/

3 2

POP3 strange logging behavior after upgrade from 2.2.13 to 2.2.25
by Iavor Stoev 18 Aug '16

18 Aug '16

Hello, After the upgrade from Dovecot version 2.2.13 to 2.2.25, we experience the following strange behavior with POP3 logging. When a connection is closed with reason "Connection closed" & "Disconnected for inactivity", the number of deleted mails is missing. Like del=/1166 & del=/4138 from the example bellow. With Dovecot 2.2.13: Feb 14 03:35:20 serverXYZ dovecot: pop3(user(a)domain.com): Connection closed top=0/0, retr=0/0, del=0/946, size=39500197 Feb 15 16:49:18 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected for inactivity top=0/0, retr=0/0, del=0/739, size=52319632 With Dovecot 2.2.25: Jul 31 02:56:03 serverXYZ dovecot: pop3(user(a)domain.com): Connection closed top=1/1380, retr=0/0, del=/4138, size=284167741 Jul 31 02:34:47 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected for inactivity top=0/0, retr=0/0, del=/1166, size=111425436 For sessions completed normally, everything works fine with both Dovecot versions: Jul 31 06:55:48 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 Jul 31 02:36:59 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected: Logged out top=0/0, retr=1/18821, del=1/1, size=18803 Please advise if this behavior is normal and if yes how should we read entries like "del=/4138"? Thank you Iavor Stoev Project Manager // Head of System & Network Administration Department ICDSoft Ltd - http://icdsoft.com

2 1

2.3.0~alpha0-1~auto+197: Crash when openening a message via IMAP
by Ralf Hildebrandt 18 Aug '16

18 Aug '16

>From the log: Jul 11 13:12:42 mproxy dovecot: imap-login: Login: user=<hildeb>, method=PLAIN, rip=141.42.206.36, lip=141.42.206.11, mpid=27254, TLS, session=<TGwoO1o3id+NKs4k> Jul 11 13:12:44 mproxy dovecot: imap(hildeb)<TGwoO1o3id+NKs4k>: Panic: file imap-client.c: line 854 (client_check_command_hangs): assertion failed: ((io_loop_find_fd_conditions(current_ioloop, client->fd_out) & IO_WRITE) != 0) Jul 11 13:12:44 mproxy dovecot: imap(hildeb)<TGwoO1o3id+NKs4k>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8e09e) [0x7efd1d3a309e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8e18c) [0x7efd1d3a318c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7efd1d34169e] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](+0x1ad42) [0x55649b35cd42] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](client_output+0x118) [0x55649b35ce68] -> /usr/lib/dovecot/libdovecot.so.0(+0xaf370) [0x7efd1d3c4370] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7efd1d3b715c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7efd1d3b85b1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7efd1d3b71e5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x30) [0x7efd1d3b7380] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7efd1d347c13] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](main+0x322) [0x55649b34faf2] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7efd1cf6c830] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](+0xdc6f) [0x55649b34fc6f] Jul 11 13:12:44 mproxy dovecot: imap(hildeb)<TGwoO1o3id+NKs4k>: Fatal: master: service(imap): child 27254 killed with signal 6 (core dumped) # 2.3.0.alpha0 (3aae884) [XI:2:2.3.0~alpha0-1~auto+197]: /etc/dovecot/dovecot.conf -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 5

dsync replication issue
by Cédric ML 18 Aug '16

18 Aug '16

Hello, I'm facing a problem with dsync : replication between my two dovecot 2.2.10 servers (10.10.10.10 = serverA and 10.10.10.11 = serverB) is not done when a mail is received on any of the two servers. Sync is correctly working when running "doveadm [-D] sync -A tcp:10.10.10.10:12345" from serverB or "doveadm [-D] sync -A tcp:10.10.10.11:12345" from serverA Sieve vacation message replication is functional too, and triggered when a change is done on any of the two servers. I guess it's a notify problem, but I did not find how to debug it... I've enabled dovecot debug, but I don't see anything related to notify in logfile. Here's my doveconf -n (it's the same on the two servers, except IP address changes) Hope somebody will be able to help me. Regards, Cédric --------------------------------------- root@dovecot1-pa3:/etc/dovecot/conf.d# doveconf -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.28.2.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_debug = yes auth_mechanisms = plain login debug_log_path = /var/log/dovecot-deliver.log doveadm_password = tgIK16l5L26023b doveadm_port = 12345 log_path = /var/log/dovecot-deliver.log mail_debug = yes mail_plugins = " quota notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vacation-seconds mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = tcp:10.10.10.10:12345 quota = maildir:User quota quota_grace = 10%% quota_rule = Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Boite de messagerie pleine. quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = ~/.dovecot.sieve sieve_before = /home/vmail/%d/%n/sieve/Vacation.sieve sieve_dir = ~/sieve sieve_extensions = +vacation-seconds } protocols = imap pop3 sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = mailbox } unix_listener replication-notify { mode = 0666 user = mailbox } } service auth { inet_listener { port = 12346 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mailbox mode = 0660 user = mailbox } } service config { unix_listener config { user = mailbox } } service doveadm { inet_listener { port = 12345 } user = mailbox } service imap-login { inet_listener imaps { port = 993 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 1 service_count = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mailbox } user = mailbox } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol lda { mail_plugins = " quota sieve quota" postmaster_address = postmaster(a)domain.fr } protocol imap { mail_plugins = " quota imap_quota" }

1 0

POP3 strange logging behavior after upgrade from 2.2.13 to 2.2.25
by George Dankin 17 Aug '16

17 Aug '16

Hello, I experience exactly the same issue. Is there any progress regarding that bug report? Cheers George Dankin Forum Leader HardwareBG.com

1 0

sieve-pipe
by zhaochangpan 16 Aug '16

16 Aug '16

hello,everyone           <span id="tran_0" data-aligning="#tran_0,#src_0" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);">I want to use the dovecot sieve - pipe plugin, I look at the website, but the plugin is not installed.<span id="tran_1" data-aligning="#tran_1,#src_1" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);">Does anyone can help me, my dovecot is 2.2.Thank you<span id="tran_2" data-aligning="#tran_2,#src_2" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);"> very much.<span id="tran_3" data-aligning="#tran_3,#src_3" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);">Reference:http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe

2 1

dsync: Keywords not being replicated
by Karsten Heiken 16 Aug '16

16 Aug '16

I just noticed that some IMAP keywords are not being replicated through dsync backup/sync using Dovecot 2.2.24 from Debian backports: On the master server everything looks fine: RECORD: seq=3, uid=44731, flags=0x09 (Seen Answered) - ext 0 keywords : (0900) <-- 0900 - ext 1 modseq : 701695 (ffb40a0000000000) - ext 3 cache : 3948 (6c0f0000) - ext 4 sort-s : 0 (00000000) On Replica: RECORD: seq=3, uid=44731, flags=0x09 (Seen Answered) - ext 0 keywords : (0000) <-- 0000 - ext 1 modseq : 701695 (ffb40a0000000000) - ext 3 cache : 3948 (6c0f0000) In this case the "forwarded" keyword is gone. Thunderbird no longer displays the message as having been forwarded. The command I used was: > doveadm sync -u myuser -l 60 -N tcp:replica:24245 My config is attached. It looks like there are other users also having the same problem: http://dovecot.org/list/dovecot/2016-January/102951.html Thanks, Karsten -- Karsten Heiken Leibniz Universität IT Services Kommunikationssysteme E-Mail, XMPP, Kalender Schloßwender Str. 5 D-30159 Hannover

1 0

[Dovecot] How to disable SSL and TLSv1.1?
by Darren Pilgrim 16 Aug '16

16 Aug '16

I'm running Dovecot 2.2.5 and want to make it refuse SSLv2, SSLv3 and TLSv1.0. Clients will opportunistically use TLS 1.1 and 1.2, but now I want require they do so. Is it enough to set ssl_cipher_list = HIGH:!SSLv2:!SSLv3:!TLSv1.0:!aNULL:!MD5 or are there additional settings I need to specify?

8 12

Segfault in lib11_trash_plugin.so
by Василий Севостьянов 16 Aug '16

16 Aug '16

After a recent update of dovecot to 2.2.devel (d81a83b) I started to get errors like that: Jul 26 07:34:44 mx kernel: pop3[17311]: segfault at 1ee74b0d5 ip 00000001ee74b0d5 sp 00007ffe593dc3c0 error 14 in lib11_trash_plugin.so[ 7f66edce2000+3000] Jul 26 08:11:25 mx dovecot: pop3(xxxxxxxx): Fatal: master: service(pop3): child 17311 killed with signal 11 (core dumps disabled) My dovecot configuration follows: # 2.2.devel (d81a83b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (3f51b82) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.5 ext4 auth_default_realm = euct.ru auth_mechanisms = plain login dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * login_greeting = Hello. mail_gid = 5000 mail_location = maildir:/home/vmail/%d/%u mail_max_userip_connections = 25 mail_plugins = " quota trash" mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = maildir:/home/vmail/%d/%%n:INDEX=/home/vmail/%d/%n/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 1 weeks special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl deleted_to_trash_folder = Trash quota = dict:User quota::proxy::quota quota_grace = 50M quota_rule = *:storage=250M quota_rule2 = Trash:storage=+50M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/sieve/.dovecot.sieve sieve_dir = ~/sieve sieve_vacation_default_period = 7d sieve_vacation_max_period = 30d sieve_vacation_min_period = 1d trash = /etc/dovecot/dovecot-trash.conf.ext } postmaster_address = postmaster(a)euct.ru protocols = " imap sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = dovecot mode = 0666 user = vmail } } service dict { unix_listener dict { group = dovecot mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl = required ssl_cert = </etc/letsencrypt/live/mail.euct.ru/fullchain.pem ssl_dh_parameters_length = 2048 ssl_key = </etc/letsencrypt/live/mail.euct.ru/privkey.pem ssl_protocols = !SSLv2 !SSLv3 userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota trash acl sieve" } protocol imap { mail_plugins = " quota trash quota imap_quota acl imap_acl deleted_to_trash" }

2 1

public namespace quota
by Chris 16 Aug '16

16 Aug '16

All, sorry, I've to ask again. I'm trying to define quotas for public namespace subfolders: ns=public/ - DepartmentA - 50G - DepartmentB - 500M - DepartmentC - 10G public/ shouldn't have a limit. public/DepartmentA is independent of B and C. Is this possible with dovecot? I've added the following rules quota = maildir:User quota quota2 = maildir:Public quota:ns=public/ quota2_rule = *:storage=10M quota2_rule2 = DepartmentA:storage=+50M but then only 10M is shown for DepartmentA in Thunderbird. I'm using Maildir++. maildirsize is only created in public/ toplevel, not in public/DepartmentA. Public-subfolders are owned by vmail system-user. Different virtual users' ACLs are set (and working). Any help is appreciated! - Chris

1 0