July 2019 - dovecot - dovecot.org

Replication Upgrade 2.2 -> 2.3 Question
by Cassidy B. Larson 16 Jul '19

16 Jul '19

Finally getting around to upgrading my setup to 2.3.x. All of our front end proxies are fine on 2.3 connecting to a 2.2 backend storage pair. So the question is, if I stage the upgrade, will it break replication between the two backend storage servers? My thought was to upgrade one to 2.3, and keep the other on 2.2 for a couple of days? If so, can 2.2 replicate to 2.3, and 2.3 replicate to 2.2 without any issue? Or am I going to have to go all-in on 2.3 at once on both? Heh. Thanks, -c

1 0

Listing expunged messages
by Mike 16 Jul '19

16 Jul '19

Hello, Following on my last question about broken mdboxes, removing dovecot.map.log and force-resync does work. My last problem now however is that after doing this, messages which were 'deleted' previously, come back in the inbox as new messages. I have the original mailboxes and can subscribe to the expunged folders and everything in EXPUNGED/INBOX on the original, is appearing in INBOX on this migrated system. I would want to return everything to EXPUNDED/INBOX so my users don't see old dead resurrected messages. Is there a procedure to identify these? Thanks. Mike-

1 0

Weird response to LIST EXTENDED
by Emil Kalchev 15 Jul '19

15 Jul '19

Hello, I was discussing weird response from Dovecot on Mailkit (this is client IMAP C# library) github page. We concluded that this is a bug in Dovecot server but I wanted to double check with you. Here are the details about the issue https://github.com/jstedfast/MailKit/issues/879 Sent with OfficeSuite Mail for Windows PC www.officesuitenow.com<https://www.officesuitenow.com>

1 0

Authdb NSS module
by Eugene 15 Jul '19

15 Jul '19

Hello! Upgrading manual tells that authdb [NSS module was removed][1] some time ago. [1]: https://wiki2.dovecot.org/Upgrading/2.3#line-100 > userdb nss was removed. Use userdb passwd instead. Can this change be reverted? I'd like to use only libnss_sss.so.2 as dovecot userdb source. It's also essential for me to enable files backend in nsswitch.conf so the system could use local user db. At the same time dovecot must not see local users at all. Authdb NSS module could help me there. The other solution would be to use another instance of nsswitch.conf for dovecot authdb passwd module. Is it possible? Thanks! -- Eugene Bright IT engineer Tel: + 79257289622

3 3

Error: o_stream_send_istream and Disconnected in APPEND
by Alessio Cecchi 15 Jul '19

15 Jul '19

Hi, I'm running some Dovecot servers configured with LVS + Director + Backend + NFS and version 2.2.36.3 (a7d78f5a2). In the last days I see an increased number of these error: Error: o_stream_send_istream(/nfs/mail/company.com/info/Maildir/.Sent/tmp/15627713… failed: Broken pipe always with action "Disconnected in APPEND", when users try to upload a message in Sent or Drafts. Here the logs: Director: Jul 10 14:53:06 imap-login: Info: proxy(info(a)company.com): started proxying to 10.0.0.20:143: user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, lport=993, TLS, session=<eA6XKFONwP1f5Sh4> Jul 10 15:09:56 imap-login: Info: proxy(info(a)company.com): disconnecting 84.1.2.3 (Disconnected by client: EOF(0s idle, in=648413, out=6602)): user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, lport=993, TLS, session=<eA6XKFONwP1f5Sh4> Backend: Jul 10 14:53:06 pop01 dovecot: imap-login: ID sent: x-session-id=eA6XKFONwP1f5Sh4, x-originating-ip=84.1.2.3, x-originating-port=64960, x-connected-ip=195.1.2.3, x-connected-port=993, x-proxy-ttl=4: user=<>, rip=84.1.2.3, lip=195.1.2.3, secured, session=<eA6XKFONwP1f5Sh4> Jul 10 14:53:06 pop01 dovecot: imap-login: Login: user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, mpid=9151, secured, session=<eA6XKFONwP1f5Sh4> Jul 10 14:53:07 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: ID sent: name=Thunderbird, version=60.7.2 Jul 10 15:09:56 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: Error: o_stream_send_istream(/nfs/mail/company.com/info/Maildir/.Sent/tmp/15627713… failed: Broken pipe Jul 10 15:09:57 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: Disconnected in APPEND (1 msgs, 48 secs, 0/2657336 bytes) in=886661 out=26654 del=1 expu=0 trash=0 I can't understand if is a network problem (Firewall? Load Balancer? Switch?) or a users LAN problem. Or others. When this happen user see an error message like "Unable to save email in Sent folder". Any suggestions? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 3

Sieve problem with duplicate and fileinto in the same set of rules
by Gianluca Scaglia 15 Jul '19

15 Jul '19

Hi there, on my mail server (postfix, dovecot 2.2.27 in Debian 9) I have an automatic forwarding (with sender_bcc_maps in Postfix) for all the emails sent in smtp from the same server, that are then put in the Sent folder with a sieve rule. In this way, however, when a user sends an e-mail to himself, both copies end up in the Sent folder and it's not good. To resolve, I tried using the Sieve "duplicate" extension along with "fileinto" but I can't get it to work. I then activated also the "editheader" extension to better understand what the problem was and I get these results. Sieve rules without using "fileinto:" require ["duplicate", "editheader"]; if duplicate { addheader "x-sieve-test1" "duplicate"; } if allof (header :regex ["from"] [".*info(a)example\.com"], header :regex ["x-sieve-test1"] [".*duplicate"]) { addheader "x-sieve-test2" "confirm duplicate"; stop; } elsif header :regex ["from"] [".*info(a)example\.com"] { addheader "x-sieve-test3" "not duplicate"; stop; } and I get these headers in the first mail (in INBOX): X-Sieve-Test2: confirm duplicate X-Sieve-Test1: duplicate and in the second email (always in INBOX) this other: X-Sieve-Test3: not duplicate Everything works perfectly. When I add the fileinto statement to the last if as below require ["duplicate", "editheader"]; if duplicate { addheader "x-sieve-test1" "duplicate"; } if allof (header :regex ["from"] [".*info(a)example\.com"], header :regex ["x-sieve-test1"] [".*duplicate"]) { addheader "x-sieve-test2" "confirm duplicate"; stop; } elsif header :regex ["from"] [".*info(a)example\.com"] { addheader "x-sieve-test3" "not duplicate"; fileinto "Sent"; stop; } I get a completely incorrect result, both mails in the Sent folder with this header line: X-Sieve-Test3: not duplicate in practice it no longer detects the duplicate email. If I use "discard" rather than "fileinto" it works fine and if I put "fileinto" in the first "if" ("if duplicate") it does't work. I did hundreds of tests but the "duplicate" extension stops working in the presence of the fileinto statement (there are exceptions: if I put the "fileinto" in the second "if" - that is in the middle one instead of the last one - it works fine, but it's not that that I want). Any idea how to solve the problem or how to get the same result (one mail in Sent folder and one in INBOX, when a user sends an e-mail to himself) without using the "duplicate" extension? Thanks a lot, Luca

2 1

Error since Dovecot v2.3.7
by Paul Hecker 15 Jul '19

15 Jul '19

Hi, since upgrading to Dovecot 2.3.7 I get the following new errors: 2019-07-15 09:10:52 mail dovecot: imap(paul(a)iwascoding.com)<32484><goWh8rKNSNZfqJ9R>: Error: file_lock_free(): Unexpectedly failed to retry locking /var/spool/mail/iwascoding/paul/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-vsize.lock: fcntl(/var/spool/mail/iwascoding/paul/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-vsize.lock, write-lock, F_SETLK) locking failed: No such file or directory It seems that this is related to moving or deleting mails (IMAP). The file at least sometimes exists, the folder exists. You find my doveconf below. Thanks, Paul # 2.3.7 (494d20bdc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7 (5a4e63b5) # OS: Linux 4.9.0-9-amd64 x86_64 Debian 9.9 # Hostname: mail.iwascoding.com auth_cache_size = 10 M auth_cache_ttl = 2 hours auth_mechanisms = plain login default_vsz_limit = 1 G first_valid_gid = 8 first_valid_uid = 8 hostname = mail.iwascoding.com imap_capability = +SPECIAL-USE last_valid_gid = 8 last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_attachment_detection_options = add-flags-on-save mail_attribute_dict = file:/var/mail/iwascoding/%n/dovecot-attributes mail_gid = 8 mail_home = /var/mail/iwascoding/%n mail_location = mdbox:~/mdbox mail_plugins = quota listescape fts fts_xapian mail_prefetch_count = 200 mail_uid = 8 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox archive { auto = no special_use = \Archive } mailbox drafts { auto = no special_use = \Drafts } mailbox junk { auto = no special_use = \Junk } mailbox sent { auto = no special_use = \Sent } mailbox trash { auto = no special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = xapian fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_autoindex_exclude3 = \Drafts fts_enforced = yes fts_xapian = partial=2 full=20 plugin = fts fts_xapian quota = count:User quota quota_grace = 10%% quota_rule = *:storage=200M quota_vsizes = yes recipient_delimiter = + sieve = /var/mail/iwascoding/%n/.dovecot.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = /var/mail/iwascoding/%n/sieve sieve_extensions = +spamtest +spamtestplus sieve_global_dir = /var/lib/dovecot/sieve/global sieve_spamtest_max_value = 200 sieve_spamtest_status_header = X-Spam-Score-Int sieve_spamtest_status_type = score } postmaster_address = postmaster(a)iwascoding.com protocols = imap pop3 lmtp sieve submission quota_full_tempfail = yes service auth { unix_listener auth-client { mode = 0600 user = Debian-exim } unix_listener auth-userdb { user = mail } vsz_limit = 4 G } service imap-login { inet_listener imap { port = 0 } } service indexer-worker { vsz_limit = 2 G } service lmtp { user = mail } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } } service stats { unix_listener stats-writer { user = mail } } service submission-login { inet_listener submission_ssl { port = 465 ssl = yes } } ssl = required ssl_alt_cert = </usr/local/acme/var/iwascoding.com/fullchain.cer ssl_alt_key = # hidden, use -P to show it ssl_cert = </usr/local/acme/var/iwascoding.com_ecc/fullchain.cer ssl_cipher_list = ALL:HIGH:!SSLv2:!SSLv3:!kRSA:!MEDIUM:!LOW:!EXP:!RC4:!MD5:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!PSK:!SRP:!DSS:!SEED:@STRENGTH ssl_client_ca_dir = /etc/ssl/certs ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes submission_client_workarounds = whitespace-before-path submission_max_mail_size = 50 M submission_relay_host = mail.iwascoding.com submission_relay_port = 10025 submission_relay_ssl = smtps userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_fsync = optimized mail_plugins = quota listescape fts fts_xapian sieve } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_fsync = optimized mail_plugins = quota listescape fts fts_xapian sieve } protocol imap { imap_metadata = yes mail_max_userip_connections = 40 mail_plugins = quota listescape fts fts_xapian imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 30 }

2 2

broken mdboxes
by Mike 15 Jul '19

15 Jul '19

Hello, I was forced to do an emergency server migration with about 3000 mailboxes and I botched it somewhat. Hoping to find some help here. My symptoms are, users showing apparently deleted mail messages, some replicated mail, and a time period of mail that is just 'missing'. The source server is centos 6.10 with dovecot 2.2.5, and the destination is ubuntu 16 / dovecot 2.2.22. User mailboxes are mdbox format and I did an rsync from the old to the new. My screw up however was that my rsync was not --delete files, and had to be run multiple times over the course of a few weeks due to to bandwidth disparities and the sheer size of the mail spool approaching 700GB (source server had 10mbps. Should have used sneakernet instead). The bottom line however is that I wound up having many extra m.* files in each user storage dir that were not actually present on the source when the final, final sync was made and I went live with the migration server. I know this is my fsckup but now I have to deal with it. I first tried to fix the symptoms with a force-resync on the boxes not realizing the above, and that seemed to make things worse by only showing new mail as of the server migration date. Ugh. New mail is being delivered and this does appear in user mail boxes. But as I said, there seems to be some users with a problem with old mail presumably deleted on the prior server suddenly re-appearing (probbly due to the extraneous m.* mdbox files present). A more pressing problem is that there is simply a time period of mail that is in user mailboxes (on the old server) from before the migration, which is not displaying now for these users on the new server. I have investigated this and hacked on "mdbox-recover.pl" somewhat so that it iterates over all mdbox files for a user and I can extract ALL messages, and see that, yes, the mail really is present in the mdbox files, but pop/imap never shows it. This is puzzling to me. I have a test user that when I check imap, it shows only 21 message in inbox and 1 in sent. But if I extract the messages from the mdbox files, then I actually have 1117 messages. I know I can't just rsync the source server again with the right --delete flag, there has been lots of mail deliveries to the migrated boxes and so I don't want to lose all of that and create yet more problems. But I need to find some way to make these present mail messages appear for my users again. Im thinking it's got to be a flag of some kind since the data is there it's just ignored. Wondering what you would reccomend? Thanks. Mike-

2 3

Unable to send the mail but able to receive them
by necktwi 15 Jul '19

15 Jul '19

From my mac mail client I've connected to remote exim+dovecot server. Since few weeks back I was not able to send mail but able to receive. exim in /var/log/messages says: Jul 14 15:22:15 RPi3B exim[3576]: 2019-07-14 20:52:15 dovecot_plain authenticator failed for ([xx.xx.xx.xx]) [yy.yy.yy.yy]: 535 Incorrect authentication data (set_id=necktwi) dovecot in /var/log/dovecot.log says: Jul 14 19:40:27 auth: Debug: client in: AUTH 1 LOGIN service=smtp rip=yy.yy.yy.yy lip=xx.xx.xx.xx nologin resp= (previous base64 data may contain sensitive data) Jul 14 19:40:27 auth: Debug: client passdb out: CONT 1 VXNlcm5hbWU6 Jul 14 19:40:27 auth: Debug: auth client connected (pid=0) Jul 14 19:40:27 auth: Debug: client in: AUTH 1 PLAIN service=smtp secured rip=xx.xx.xx.xx lip=yy.yy.yy.yy nologin resp=aeVja333aQeuZWerdHdpAeRecmezaG9lcw== (previous base64 data may contain sensitive data) Jul 14 19:40:27 auth: Debug: policy(necktwi,xx.xx.xx.xx): Policy check action is continue Jul 14 19:40:27 auth-worker(11866): Debug: pam(necktwi,xx.xx.xx.xx): lookup service=smtp Jul 14 19:40:27 auth-worker(11866): Info: pam(necktwi,xx.xx.xx.xx): pam_authenticate() failed: Authentication failure (Password mismatch?) (given password: xxxxxxx) (log is obfuscated) Is it the problem with dovecot or exim? Should exim contact dovecot prior to sending the mail? I have also tried windows mail client with same error. auth-system.conf.ext: passdb { driver = pam args = %s } userdb { driver = passwd args = blocking=no } ... Necktwi

1 0

Sent messages disappear and don't arrive. (Receiving is ok)
by John 15 Jul '19

15 Jul '19

Hi List, I'm running an email server (Postfix, Dovecot, MySQL) on an RPi. I set it up using this 'howto' https://pestmeester.nl/index.html#11.0. It seemed to work but now sent emails just disappear and are not received. Inward emails are ok. I'm a Linux and Postfix Dovecot learner so please treat me as not being very knowledgeable. I'm not even sure if this is a Postfix or Dovecot issue so I've posted to both. (Nothing from Postfix so far!) I've tried to include what relevant info I thought a helper might need but if I've missed something please let me know. I'll appreciate any help. TIA, John

3 2