August 2016 - dovecot

What might cause messages delivered with dovecot-lda to be "invisible" to IMAP clients?
by Ben Johnson 19 Aug '16

19 Aug '16

Hello! I'm attempting to use dovecot-lda, and for whatever reason, messages that I send with it are not visible in my IMAP client. $ dovecot --version 2.2.22 (fe789d2) (Full "doveconf -n" output is at the bottom of this message!) Here's the the sequence of commands that I'm using: # su vmail $ echo 'This is a test message.' > /tmp/test.txt $ stat /tmp/test.txt File: '/tmp/test.txt' Size: 24 Blocks: 8 IO Block: 4096 regular file Device: fd01h/64769d Inode: 783925 Links: 2 Access: (0664/-rw-rw-r--) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2016-08-18 20:49:38.419011552 -0400 Modify: 2016-08-18 20:49:07.178995015 -0400 Change: 2016-08-18 20:49:38.191011425 -0400 Birth: - $ /usr/lib/dovecot/dovecot-lda -d "test(a)example.com" -p "/tmp/test.txt" When I do this, the destination mailbox acts like there's a new message (I'm using Thunderbird, and the mailbox is highlighted in blue and bolded, which is the "new mail" behavior in this client). But when I "check my inbox", i.e., click on the Inbox, I don't see the new message. With regard to the filesystem, the message is present and contains the intended contents. Here's the "stat" output: $ stat /var/vmail/example.com/ben/Maildir/cur/1471567777.M685753P32257.example.com,S=24:2, File: '/var/vmail/example.com/ben/Maildir/cur/1471567777.M685753P32257.example.com,S=24:2,' Size: 24 Blocks: 8 IO Block: 4096 regular file Device: fd01h/64769d Inode: 783925 Links: 2 Access: (0664/-rw-rw-r--) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2016-08-18 20:51:05.647059340 -0400 Modify: 2016-08-18 20:50:58.543055378 -0400 Change: 2016-08-18 20:50:58.543055378 -0400 Birth: - Might anyone know why I'm not "seeing" this message in my IMAP client? Interestingly, I tried another IMAP client (Blue Mail for Android), and I can actually see the messages there, but they appear in the Inbox as: (No subject) <Empty Message> When I click on the messages, Blue Mail simply displays "No text", without the quotes. Surely, I'm overlooking something simple... Thank you for any help here! -Ben Full "doveconf -n" output: $ doveconf -n # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS auth_mechanisms = plain login disable_plaintext_auth = no listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_max_userip_connections = 100 mail_plugins = " quota" mail_privileged_group = vmail passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /bin/bash antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = /tmp antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_trash_pattern_ignorecase = trash;Deleted * antispam_verbose_debug = 1 quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_rule2 = Trash:storage=+100M quota_rule3 = Junk:ignore quota_rule4 = INBOX:storage=+100M quota_warning = storage=100%% quota-reached 100 %u %d quota_warning2 = storage=95%% quota-warning 95 %u %d quota_warning3 = storage=80%% quota-warning 80 %u %d quota_warning4 = -storage=100%% quota-below below %u %d sieve = /var/vmail/%d/%n/.sieve sieve_max_redirects = 25 } postmaster_address = postmaster(a)example.com protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 512 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-below { executable = script /usr/local/bin/quota-below.sh unix_listener quota-below { group = vmail mode = 0666 user = vmail } user = vmail } service quota-reached { executable = script /usr/local/bin/quota-reached.sh unix_listener quota-reached { group = vmail mode = 0666 user = vmail } user = vmail } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol imap { mail_plugins = quota imap_quota antispam } protocol pop3 { mail_plugins = quota pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve quota } protocol lmtp { mail_plugins = quota sieve postmaster_address = webmaster@localhost }

1 1

Quota issue: enforced, but reporting issue
by dovecot＠avv.solutions 18 Aug '16

18 Aug '16

Hello everyone, I face the following challenge: - dovecot 2.2.13 - userdb: passwd-file - quota enabled: -- default 1GB -- changed for some users thru users file: user@domain:passwd::::::userdb_quota_rule=*:storage=15M - when sending a message to this user while near quota, warnings are correctly triggered => *OK* - when sending a message to this user while over quota, message is refused => *OK* - when checking quota from mail client (eg Thunderbird 45.2), reported quota is.. 1GB => *NOK* - some config: mail_plugins = quota notify acl plugin { acl_shared_dict = file:/sd/MAIL_IMAP_POP/%d/shared-mailboxes quota = maildir:User quota:ns= quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=1G quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO quota_warning = storage=90%% quota-warning 90 %u %d quota_warning2 = storage=75%% quota-warning 75 %u %d } protocol imap { mail_plugins = quota notify acl imap_quota imap_acl service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } } - 90-quota: passdb { driver = passwd-file args = scheme=CRYPT username_format=%u /etc/dovecot/users } userdb { driver = passwd-file args = username_format=%u /etc/dovecot/users # Default fields that can be overridden by passwd-file #default_fields = quota_rule=*:storage=1G # Override fields from passwd-file #override_fields = home=/home/virtual/%u } Any idea? Already spent hours on this. Thank you in advance for the help. Cheers A.

1 0

Reporting on CephFS being ready to use with Dovecot
by Daniel van Ham Colchete 18 Aug '16

18 Aug '16

I would like to report that from version 10.2.3 on (next release), Ceph FS is working really well with Dovecot systems. For those that don't know, Ceph is a "distributed object store and file system designed to provide excellent performance, reliability and scalability.". We have used it here since 2013 very successfully, but never with our Dovecot setup. For more information go on http://ceph.com/. Since Ceph Jewel (the current version), Ceph FS is considered production ready by their team. With Ceph FS you have a cache-coherent POSIX-compliant [1] clustered file system, without most of the NFS shortcomings. Ceph have very nice features like online upgrades, online maintenance, constant deep scrubbing of replicated data, cache tiering (HD -> SSD -> etc), erasure coding (clustered RAID6 for really old email - I'm not using), etc. Ceph is very complex to operate but very flexible and robust. This year we moved our Dovecot servers to a Ceph based system, found one bug there (http://tracker.ceph.com/issues/15920) when Dovecot's LMTP was delivery an email, and the fix is about to be released on version 10.2.3. I have been using a fix-build here for a couple of months without issue. So, now I'm glad to share with you guys that it works really well! My setup involves two clusters, each with about 30k-40k users. Each cluster will have two HD storages (with 6TB HDs), two SSD storages (with Intel 480GB SSDs) and two frontends. In a few months will add a third server of each type. Clusters work better with 3's. Here we used mdbox as it is better performant on Ceph for maintenance tasks, since each file is an object on Ceph and maintenance costs increase with the number of objects. We created two base directories: - /srv/dovecot/mail/%d/%n - stored on HDs with the most recent files cached on SSDs, thanks to Ceph Cache Tiering. Also, the directory structure itself is stored on SSDs, so dir listings are very fast (Ceph FS Metadata). - /srv/dovecot/index/%d/%n - stored only on SSDs, thanks for Ceph FS file layout. On our setup about 17% of the IOPs are going to HDs, the rest will go to SSDs, even though SSDs are less than 5% of the space. This is a matter of tuning the cache tiering parameters, but we didn't look at that yet. That setup is working like a charm, performance is about 53% better than when we were using NFS on the same hardware. Our previous DRBD+Heartbeat setup didn't allow for online maintenance and had a few problems. Now we can do 100% online maintenance on storage without users noticing, and on frontends with just a reconnect but without any downtime. Ceph is hard to learn at first but those with bigger setups and stronger SLAs will want to take a look at that. I really recommend that the Dovecot community take at look at that setup. Good luck! Best, Daniel Colchete [1] http://docs.ceph.com/docs/hammer/dev/differences-from-posix/

3 2

POP3 strange logging behavior after upgrade from 2.2.13 to 2.2.25
by Iavor Stoev 18 Aug '16

18 Aug '16

Hello, After the upgrade from Dovecot version 2.2.13 to 2.2.25, we experience the following strange behavior with POP3 logging. When a connection is closed with reason "Connection closed" & "Disconnected for inactivity", the number of deleted mails is missing. Like del=/1166 & del=/4138 from the example bellow. With Dovecot 2.2.13: Feb 14 03:35:20 serverXYZ dovecot: pop3(user(a)domain.com): Connection closed top=0/0, retr=0/0, del=0/946, size=39500197 Feb 15 16:49:18 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected for inactivity top=0/0, retr=0/0, del=0/739, size=52319632 With Dovecot 2.2.25: Jul 31 02:56:03 serverXYZ dovecot: pop3(user(a)domain.com): Connection closed top=1/1380, retr=0/0, del=/4138, size=284167741 Jul 31 02:34:47 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected for inactivity top=0/0, retr=0/0, del=/1166, size=111425436 For sessions completed normally, everything works fine with both Dovecot versions: Jul 31 06:55:48 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 Jul 31 02:36:59 serverXYZ dovecot: pop3(user(a)domain.com): Disconnected: Logged out top=0/0, retr=1/18821, del=1/1, size=18803 Please advise if this behavior is normal and if yes how should we read entries like "del=/4138"? Thank you Iavor Stoev Project Manager // Head of System & Network Administration Department ICDSoft Ltd - http://icdsoft.com

2 1

2.3.0~alpha0-1~auto+197: Crash when openening a message via IMAP
by Ralf Hildebrandt 18 Aug '16

18 Aug '16

>From the log: Jul 11 13:12:42 mproxy dovecot: imap-login: Login: user=<hildeb>, method=PLAIN, rip=141.42.206.36, lip=141.42.206.11, mpid=27254, TLS, session=<TGwoO1o3id+NKs4k> Jul 11 13:12:44 mproxy dovecot: imap(hildeb)<TGwoO1o3id+NKs4k>: Panic: file imap-client.c: line 854 (client_check_command_hangs): assertion failed: ((io_loop_find_fd_conditions(current_ioloop, client->fd_out) & IO_WRITE) != 0) Jul 11 13:12:44 mproxy dovecot: imap(hildeb)<TGwoO1o3id+NKs4k>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8e09e) [0x7efd1d3a309e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8e18c) [0x7efd1d3a318c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7efd1d34169e] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](+0x1ad42) [0x55649b35cd42] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](client_output+0x118) [0x55649b35ce68] -> /usr/lib/dovecot/libdovecot.so.0(+0xaf370) [0x7efd1d3c4370] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7efd1d3b715c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7efd1d3b85b1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7efd1d3b71e5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x30) [0x7efd1d3b7380] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7efd1d347c13] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](main+0x322) [0x55649b34faf2] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7efd1cf6c830] -> dovecot/imap [hildeb 141.42.206.36 UID FETCH - 8192 bytes waiting corked](+0xdc6f) [0x55649b34fc6f] Jul 11 13:12:44 mproxy dovecot: imap(hildeb)<TGwoO1o3id+NKs4k>: Fatal: master: service(imap): child 27254 killed with signal 6 (core dumped) # 2.3.0.alpha0 (3aae884) [XI:2:2.3.0~alpha0-1~auto+197]: /etc/dovecot/dovecot.conf -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

3 5

dsync replication issue
by Cédric ML 18 Aug '16

18 Aug '16

Hello, I'm facing a problem with dsync : replication between my two dovecot 2.2.10 servers (10.10.10.10 = serverA and 10.10.10.11 = serverB) is not done when a mail is received on any of the two servers. Sync is correctly working when running "doveadm [-D] sync -A tcp:10.10.10.10:12345" from serverB or "doveadm [-D] sync -A tcp:10.10.10.11:12345" from serverA Sieve vacation message replication is functional too, and triggered when a change is done on any of the two servers. I guess it's a notify problem, but I did not find how to debug it... I've enabled dovecot debug, but I don't see anything related to notify in logfile. Here's my doveconf -n (it's the same on the two servers, except IP address changes) Hope somebody will be able to help me. Regards, Cédric --------------------------------------- root@dovecot1-pa3:/etc/dovecot/conf.d# doveconf -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.28.2.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) auth_debug = yes auth_mechanisms = plain login debug_log_path = /var/log/dovecot-deliver.log doveadm_password = tgIK16l5L26023b doveadm_port = 12345 log_path = /var/log/dovecot-deliver.log mail_debug = yes mail_plugins = " quota notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vacation-seconds mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = tcp:10.10.10.10:12345 quota = maildir:User quota quota_grace = 10%% quota_rule = Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Boite de messagerie pleine. quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u sieve = ~/.dovecot.sieve sieve_before = /home/vmail/%d/%n/sieve/Vacation.sieve sieve_dir = ~/sieve sieve_extensions = +vacation-seconds } protocols = imap pop3 sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = mailbox } unix_listener replication-notify { mode = 0666 user = mailbox } } service auth { inet_listener { port = 12346 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mailbox mode = 0660 user = mailbox } } service config { unix_listener config { user = mailbox } } service doveadm { inet_listener { port = 12345 } user = mailbox } service imap-login { inet_listener imaps { port = 993 } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 1 service_count = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mailbox } user = mailbox } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol lda { mail_plugins = " quota sieve quota" postmaster_address = postmaster(a)domain.fr } protocol imap { mail_plugins = " quota imap_quota" }

1 0

POP3 strange logging behavior after upgrade from 2.2.13 to 2.2.25
by George Dankin 17 Aug '16

17 Aug '16

Hello, I experience exactly the same issue. Is there any progress regarding that bug report? Cheers George Dankin Forum Leader HardwareBG.com

1 0

sieve-pipe
by zhaochangpan 16 Aug '16

16 Aug '16

hello,everyone           <span id="tran_0" data-aligning="#tran_0,#src_0" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);">I want to use the dovecot sieve - pipe plugin, I look at the website, but the plugin is not installed.<span id="tran_1" data-aligning="#tran_1,#src_1" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);">Does anyone can help me, my dovecot is 2.2.Thank you<span id="tran_2" data-aligning="#tran_2,#src_2" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);"> very much.<span id="tran_3" data-aligning="#tran_3,#src_3" class="copied" style="margin: 0px; padding: 0px; border: 0px; outline: 0px; color: rgb(102, 102, 102); font-family: Tahoma, Arial, 宋体, 'Malgun Gothic'; line-height: 24px; text-align: justify; widows: auto; background-color: rgba(255, 255, 255, 0.8);">Reference:http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Pipe

2 1

dsync: Keywords not being replicated
by Karsten Heiken 16 Aug '16

16 Aug '16

I just noticed that some IMAP keywords are not being replicated through dsync backup/sync using Dovecot 2.2.24 from Debian backports: On the master server everything looks fine: RECORD: seq=3, uid=44731, flags=0x09 (Seen Answered) - ext 0 keywords : (0900) <-- 0900 - ext 1 modseq : 701695 (ffb40a0000000000) - ext 3 cache : 3948 (6c0f0000) - ext 4 sort-s : 0 (00000000) On Replica: RECORD: seq=3, uid=44731, flags=0x09 (Seen Answered) - ext 0 keywords : (0000) <-- 0000 - ext 1 modseq : 701695 (ffb40a0000000000) - ext 3 cache : 3948 (6c0f0000) In this case the "forwarded" keyword is gone. Thunderbird no longer displays the message as having been forwarded. The command I used was: > doveadm sync -u myuser -l 60 -N tcp:replica:24245 My config is attached. It looks like there are other users also having the same problem: http://dovecot.org/list/dovecot/2016-January/102951.html Thanks, Karsten -- Karsten Heiken Leibniz Universität IT Services Kommunikationssysteme E-Mail, XMPP, Kalender Schloßwender Str. 5 D-30159 Hannover

1 0

[Dovecot] How to disable SSL and TLSv1.1?
by Darren Pilgrim 16 Aug '16

16 Aug '16

I'm running Dovecot 2.2.5 and want to make it refuse SSLv2, SSLv3 and TLSv1.0. Clients will opportunistically use TLS 1.1 and 1.2, but now I want require they do so. Is it enough to set ssl_cipher_list = HIGH:!SSLv2:!SSLv3:!TLSv1.0:!aNULL:!MD5 or are there additional settings I need to specify?

8 12