July 2019 - dovecot - dovecot.org

Error: o_stream_send_istream and Disconnected in APPEND
by Alessio Cecchi 15 Jul '19

15 Jul '19

Hi, I'm running some Dovecot servers configured with LVS + Director + Backend + NFS and version 2.2.36.3 (a7d78f5a2). In the last days I see an increased number of these error: Error: o_stream_send_istream(/nfs/mail/company.com/info/Maildir/.Sent/tmp/15627713… failed: Broken pipe always with action "Disconnected in APPEND", when users try to upload a message in Sent or Drafts. Here the logs: Director: Jul 10 14:53:06 imap-login: Info: proxy(info(a)company.com): started proxying to 10.0.0.20:143: user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, lport=993, TLS, session=<eA6XKFONwP1f5Sh4> Jul 10 15:09:56 imap-login: Info: proxy(info(a)company.com): disconnecting 84.1.2.3 (Disconnected by client: EOF(0s idle, in=648413, out=6602)): user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, lport=993, TLS, session=<eA6XKFONwP1f5Sh4> Backend: Jul 10 14:53:06 pop01 dovecot: imap-login: ID sent: x-session-id=eA6XKFONwP1f5Sh4, x-originating-ip=84.1.2.3, x-originating-port=64960, x-connected-ip=195.1.2.3, x-connected-port=993, x-proxy-ttl=4: user=<>, rip=84.1.2.3, lip=195.1.2.3, secured, session=<eA6XKFONwP1f5Sh4> Jul 10 14:53:06 pop01 dovecot: imap-login: Login: user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, mpid=9151, secured, session=<eA6XKFONwP1f5Sh4> Jul 10 14:53:07 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: ID sent: name=Thunderbird, version=60.7.2 Jul 10 15:09:56 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: Error: o_stream_send_istream(/nfs/mail/company.com/info/Maildir/.Sent/tmp/15627713… failed: Broken pipe Jul 10 15:09:57 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: Disconnected in APPEND (1 msgs, 48 secs, 0/2657336 bytes) in=886661 out=26654 del=1 expu=0 trash=0 I can't understand if is a network problem (Firewall? Load Balancer? Switch?) or a users LAN problem. Or others. When this happen user see an error message like "Unable to save email in Sent folder". Any suggestions? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 3

Sieve problem with duplicate and fileinto in the same set of rules
by Gianluca Scaglia 15 Jul '19

15 Jul '19

Hi there, on my mail server (postfix, dovecot 2.2.27 in Debian 9) I have an automatic forwarding (with sender_bcc_maps in Postfix) for all the emails sent in smtp from the same server, that are then put in the Sent folder with a sieve rule. In this way, however, when a user sends an e-mail to himself, both copies end up in the Sent folder and it's not good. To resolve, I tried using the Sieve "duplicate" extension along with "fileinto" but I can't get it to work. I then activated also the "editheader" extension to better understand what the problem was and I get these results. Sieve rules without using "fileinto:" require ["duplicate", "editheader"]; if duplicate { addheader "x-sieve-test1" "duplicate"; } if allof (header :regex ["from"] [".*info(a)example\.com"], header :regex ["x-sieve-test1"] [".*duplicate"]) { addheader "x-sieve-test2" "confirm duplicate"; stop; } elsif header :regex ["from"] [".*info(a)example\.com"] { addheader "x-sieve-test3" "not duplicate"; stop; } and I get these headers in the first mail (in INBOX): X-Sieve-Test2: confirm duplicate X-Sieve-Test1: duplicate and in the second email (always in INBOX) this other: X-Sieve-Test3: not duplicate Everything works perfectly. When I add the fileinto statement to the last if as below require ["duplicate", "editheader"]; if duplicate { addheader "x-sieve-test1" "duplicate"; } if allof (header :regex ["from"] [".*info(a)example\.com"], header :regex ["x-sieve-test1"] [".*duplicate"]) { addheader "x-sieve-test2" "confirm duplicate"; stop; } elsif header :regex ["from"] [".*info(a)example\.com"] { addheader "x-sieve-test3" "not duplicate"; fileinto "Sent"; stop; } I get a completely incorrect result, both mails in the Sent folder with this header line: X-Sieve-Test3: not duplicate in practice it no longer detects the duplicate email. If I use "discard" rather than "fileinto" it works fine and if I put "fileinto" in the first "if" ("if duplicate") it does't work. I did hundreds of tests but the "duplicate" extension stops working in the presence of the fileinto statement (there are exceptions: if I put the "fileinto" in the second "if" - that is in the middle one instead of the last one - it works fine, but it's not that that I want). Any idea how to solve the problem or how to get the same result (one mail in Sent folder and one in INBOX, when a user sends an e-mail to himself) without using the "duplicate" extension? Thanks a lot, Luca

2 1

Error since Dovecot v2.3.7
by Paul Hecker 15 Jul '19

15 Jul '19

Hi, since upgrading to Dovecot 2.3.7 I get the following new errors: 2019-07-15 09:10:52 mail dovecot: imap(paul(a)iwascoding.com)<32484><goWh8rKNSNZfqJ9R>: Error: file_lock_free(): Unexpectedly failed to retry locking /var/spool/mail/iwascoding/paul/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-vsize.lock: fcntl(/var/spool/mail/iwascoding/paul/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-vsize.lock, write-lock, F_SETLK) locking failed: No such file or directory It seems that this is related to moving or deleting mails (IMAP). The file at least sometimes exists, the folder exists. You find my doveconf below. Thanks, Paul # 2.3.7 (494d20bdc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7 (5a4e63b5) # OS: Linux 4.9.0-9-amd64 x86_64 Debian 9.9 # Hostname: mail.iwascoding.com auth_cache_size = 10 M auth_cache_ttl = 2 hours auth_mechanisms = plain login default_vsz_limit = 1 G first_valid_gid = 8 first_valid_uid = 8 hostname = mail.iwascoding.com imap_capability = +SPECIAL-USE last_valid_gid = 8 last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_attachment_detection_options = add-flags-on-save mail_attribute_dict = file:/var/mail/iwascoding/%n/dovecot-attributes mail_gid = 8 mail_home = /var/mail/iwascoding/%n mail_location = mdbox:~/mdbox mail_plugins = quota listescape fts fts_xapian mail_prefetch_count = 200 mail_uid = 8 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox archive { auto = no special_use = \Archive } mailbox drafts { auto = no special_use = \Drafts } mailbox junk { auto = no special_use = \Junk } mailbox sent { auto = no special_use = \Sent } mailbox trash { auto = no special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = xapian fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_autoindex_exclude3 = \Drafts fts_enforced = yes fts_xapian = partial=2 full=20 plugin = fts fts_xapian quota = count:User quota quota_grace = 10%% quota_rule = *:storage=200M quota_vsizes = yes recipient_delimiter = + sieve = /var/mail/iwascoding/%n/.dovecot.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = /var/mail/iwascoding/%n/sieve sieve_extensions = +spamtest +spamtestplus sieve_global_dir = /var/lib/dovecot/sieve/global sieve_spamtest_max_value = 200 sieve_spamtest_status_header = X-Spam-Score-Int sieve_spamtest_status_type = score } postmaster_address = postmaster(a)iwascoding.com protocols = imap pop3 lmtp sieve submission quota_full_tempfail = yes service auth { unix_listener auth-client { mode = 0600 user = Debian-exim } unix_listener auth-userdb { user = mail } vsz_limit = 4 G } service imap-login { inet_listener imap { port = 0 } } service indexer-worker { vsz_limit = 2 G } service lmtp { user = mail } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } } service stats { unix_listener stats-writer { user = mail } } service submission-login { inet_listener submission_ssl { port = 465 ssl = yes } } ssl = required ssl_alt_cert = </usr/local/acme/var/iwascoding.com/fullchain.cer ssl_alt_key = # hidden, use -P to show it ssl_cert = </usr/local/acme/var/iwascoding.com_ecc/fullchain.cer ssl_cipher_list = ALL:HIGH:!SSLv2:!SSLv3:!kRSA:!MEDIUM:!LOW:!EXP:!RC4:!MD5:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!PSK:!SRP:!DSS:!SEED:@STRENGTH ssl_client_ca_dir = /etc/ssl/certs ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes submission_client_workarounds = whitespace-before-path submission_max_mail_size = 50 M submission_relay_host = mail.iwascoding.com submission_relay_port = 10025 submission_relay_ssl = smtps userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_fsync = optimized mail_plugins = quota listescape fts fts_xapian sieve } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_fsync = optimized mail_plugins = quota listescape fts fts_xapian sieve } protocol imap { imap_metadata = yes mail_max_userip_connections = 40 mail_plugins = quota listescape fts fts_xapian imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 30 }

2 2

broken mdboxes
by Mike 15 Jul '19

15 Jul '19

Hello, I was forced to do an emergency server migration with about 3000 mailboxes and I botched it somewhat. Hoping to find some help here. My symptoms are, users showing apparently deleted mail messages, some replicated mail, and a time period of mail that is just 'missing'. The source server is centos 6.10 with dovecot 2.2.5, and the destination is ubuntu 16 / dovecot 2.2.22. User mailboxes are mdbox format and I did an rsync from the old to the new. My screw up however was that my rsync was not --delete files, and had to be run multiple times over the course of a few weeks due to to bandwidth disparities and the sheer size of the mail spool approaching 700GB (source server had 10mbps. Should have used sneakernet instead). The bottom line however is that I wound up having many extra m.* files in each user storage dir that were not actually present on the source when the final, final sync was made and I went live with the migration server. I know this is my fsckup but now I have to deal with it. I first tried to fix the symptoms with a force-resync on the boxes not realizing the above, and that seemed to make things worse by only showing new mail as of the server migration date. Ugh. New mail is being delivered and this does appear in user mail boxes. But as I said, there seems to be some users with a problem with old mail presumably deleted on the prior server suddenly re-appearing (probbly due to the extraneous m.* mdbox files present). A more pressing problem is that there is simply a time period of mail that is in user mailboxes (on the old server) from before the migration, which is not displaying now for these users on the new server. I have investigated this and hacked on "mdbox-recover.pl" somewhat so that it iterates over all mdbox files for a user and I can extract ALL messages, and see that, yes, the mail really is present in the mdbox files, but pop/imap never shows it. This is puzzling to me. I have a test user that when I check imap, it shows only 21 message in inbox and 1 in sent. But if I extract the messages from the mdbox files, then I actually have 1117 messages. I know I can't just rsync the source server again with the right --delete flag, there has been lots of mail deliveries to the migrated boxes and so I don't want to lose all of that and create yet more problems. But I need to find some way to make these present mail messages appear for my users again. Im thinking it's got to be a flag of some kind since the data is there it's just ignored. Wondering what you would reccomend? Thanks. Mike-

2 3

Unable to send the mail but able to receive them
by necktwi 15 Jul '19

15 Jul '19

From my mac mail client I've connected to remote exim+dovecot server. Since few weeks back I was not able to send mail but able to receive. exim in /var/log/messages says: Jul 14 15:22:15 RPi3B exim[3576]: 2019-07-14 20:52:15 dovecot_plain authenticator failed for ([xx.xx.xx.xx]) [yy.yy.yy.yy]: 535 Incorrect authentication data (set_id=necktwi) dovecot in /var/log/dovecot.log says: Jul 14 19:40:27 auth: Debug: client in: AUTH 1 LOGIN service=smtp rip=yy.yy.yy.yy lip=xx.xx.xx.xx nologin resp= (previous base64 data may contain sensitive data) Jul 14 19:40:27 auth: Debug: client passdb out: CONT 1 VXNlcm5hbWU6 Jul 14 19:40:27 auth: Debug: auth client connected (pid=0) Jul 14 19:40:27 auth: Debug: client in: AUTH 1 PLAIN service=smtp secured rip=xx.xx.xx.xx lip=yy.yy.yy.yy nologin resp=aeVja333aQeuZWerdHdpAeRecmezaG9lcw== (previous base64 data may contain sensitive data) Jul 14 19:40:27 auth: Debug: policy(necktwi,xx.xx.xx.xx): Policy check action is continue Jul 14 19:40:27 auth-worker(11866): Debug: pam(necktwi,xx.xx.xx.xx): lookup service=smtp Jul 14 19:40:27 auth-worker(11866): Info: pam(necktwi,xx.xx.xx.xx): pam_authenticate() failed: Authentication failure (Password mismatch?) (given password: xxxxxxx) (log is obfuscated) Is it the problem with dovecot or exim? Should exim contact dovecot prior to sending the mail? I have also tried windows mail client with same error. auth-system.conf.ext: passdb { driver = pam args = %s } userdb { driver = passwd args = blocking=no } ... Necktwi

1 0

Sent messages disappear and don't arrive. (Receiving is ok)
by John 14 Jul '19

14 Jul '19

Hi List, I'm running an email server (Postfix, Dovecot, MySQL) on an RPi. I set it up using this 'howto' https://pestmeester.nl/index.html#11.0. It seemed to work but now sent emails just disappear and are not received. Inward emails are ok. I'm a Linux and Postfix Dovecot learner so please treat me as not being very knowledgeable. I'm not even sure if this is a Postfix or Dovecot issue so I've posted to both. (Nothing from Postfix so far!) I've tried to include what relevant info I thought a helper might need but if I've missed something please let me know. I'll appreciate any help. TIA, John

3 2

Purpose of stats-writer and why doveadm try to open it to dump stats ?
by Jean-Daniel 14 Jul '19

14 Jul '19

Hello, I want to monitor dovecot stats, and so I have an exporter process that run with limited rights. The monitoring user has only access to /var/run/dovecot/stats-reader and it works fine. Doveadm stats dump returns the list of all stats as expected. But each time I run doveadm stats dump, it logs the following error: Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied So what is the purpose of the stats-writer socket, and why doveadm try to open it to simply dump stats ? Is it really something it needs and I should update my user permissions or is it a doveadm bug ?

2 1

Re: Purpose of stats-writer and why doveadm try to open it to dump stats ?
by Jean-Daniel 14 Jul '19

14 Jul '19

> Le 14 juil. 2019 à 11:58, Reio Remma <reio(a)mrstuudio.ee> a écrit : > > On 14.07.2019 10:10, Jean-Daniel via dovecot wrote: >> Hello, >> >> I want to monitor dovecot stats, and so I have an exporter process that run with limited rights. >> The monitoring user has only access to /var/run/dovecot/stats-reader and it works fine. >> Doveadm stats dump returns the list of all stats as expected. >> >> But each time I run doveadm stats dump, it logs the following error: >> >> Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied >> >> So what is the purpose of the stats-writer socket, and why doveadm try to open it to simply dump stats ? >> Is it really something it needs and I should update my user permissions or is it a doveadm bug ? >> > > Hello! > > Depending on the system you're running on, there may be SELinux etc. doing its work. > I know why stats-writer is not accessible, this is on purpose (doveadm run with a uid that explicitly don’t have access to stats-writer). What I want to know is why does doveadm even try to open it to simply dump the stats.

1 0

Getting SSL certificate/key from database
by Yevgeny Kosarzhevsky 14 Jul '19

14 Jul '19

Hello, I am not finding how to read SSL keys/certificates from database. Is this possible? Or only file reads allowed? -- Regards, Yevgeny

2 1

Pigeonhole-0.5.7 fails to build from sources
by Lefteris Tsintjelis 13 Jul '19

13 Jul '19

Latest update fails to build from sources in FreeBSD 12-STABLE --- lda-sieve-plugin.lo --- lda-sieve-plugin.c:82:27: error: variable has incomplete type 'struct smtp_submit_input' struct smtp_submit_input submit_input; ^ lda-sieve-plugin.c:82:9: note: forward declaration of 'struct smtp_submit_input' struct smtp_submit_input submit_input; ^ 1 error generated. *** [lda-sieve-plugin.lo] Error code 1 make[6]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins/lda-sieve --- lda-sieve-log.lo --- mv -f .deps/lda-sieve-log.Tpo .deps/lda-sieve-log.Plo 1 error make[6]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins/lda-sieve *** [all-recursive] Error code 1 make[5]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins 1 error make[5]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins *** [all-recursive] Error code 1 make[4]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src 1 error make[4]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src *** [all-recursive] Error code 1 make[3]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 1 error make[3]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 *** [all] Error code 2 make[2]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 1 error make[2]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 ===> Compilation failed unexpectedly. Ensure that dovecot-pigeonhole's GSSAPI backend is the same as dovecot2 (make config). Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *** Error code 1 Stop. make[1]: stopped in /usr/ports/mail/dovecot-pigeonhole *** Error code 1 Stop. make: stopped in /usr/ports/mail/dovecot-pigeonhole

2 3