From skdovecot at smail.inf.fh-brs.de Tue Dec 1 07:41:30 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 1 Dec 2015 08:41:30 +0100 (CET) Subject: Sieve, shared folders, different daemons In-Reply-To: <565C5DCD.1090706@0tten.nl> References: <565C5DCD.1090706@0tten.nl> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 30 Nov 2015, Frido Otten wrote: > In our dovecot setup we use 3 different config files. One for > maildropping(sieve), one for dovecot/imap on standard ports with ssl > config and one for dovecot/imap on standard ports+1 with a different ssl > certificate. Currently the client using the standard port+1 config wants > to make use of shared folders, but the thing is that the current > namespace separator in all configs is '.'. This is conflicting with > shared folders. How does '.' conflict with shared folders? > Can we simply run one of the dovecot/imap daemons > configured with a different separator and shared folders without having > impact on sieve and the current connected users to the standard ports? I hate all questions and answers with "simple" or "simply" in it. But yes, the IMAP separator does not have no impact on the filesystem _usually_. But mayhap you have a strange config anyway (with 3 configs) and the separator does influence the storage. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVl1PKnz1H7kL/d9rAQKg1Qf+N0M28bjIXatRlYIlk7AG99DN5pvN/ZhF X70oflJFc947fybOIiiTvvnz7yQJNtsp8BQ1k0GJ5GWB7vgyEhOzgrVykkB9QErQ Teme5zSQiG8KS8E8rDDatRn2RdQ/7jhAeALKEzWvjRXkz3zDqqiblkPnH/7vjxho zwaBfcH8AGhkhTE0st8Z3jxlaIB0wdjWtZYKZCD3jnQziurqae0o4HD4IcGSROpo EY5PzLN8FROzS9t2dowm/meCTP1+jy+is06weALlw1fzUTglD2firn5OPESsXcFG Hl2mL2Jt3wsp/m+nVv8XyK5ZTA5zE5IP5djjBxcT3SbBtPmaXOB7ig== =qeFK -----END PGP SIGNATURE----- From alessio at skye.it Tue Dec 1 09:36:30 2015 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 1 Dec 2015 10:36:30 +0100 Subject: mail_log plugin and uid=error in log file Message-ID: <565D6A1E.9090405@skye.it> Hi, I enabled notify and mail_log plugins to log the message uid for new emails delivered via LMTP. All works fine except that the uid is not always logged (see uid=error instead of uid=NUMBER) from dovecot.log: Dec 01 10:03:08 lmtp(alessio at domain.com): Info: copy from : box=INBOX, uid=error, msgid=<14489594 at domain.com>, from="WordPress" , subject=Enquiry from Joy Dec 01 10:03:09 lmtp(alessio at domain.com): Info: copy from : box=INBOX, uid=92, msgid=<03c4e16d at www.domain.com>, from="WordPress" , subject=Enquiry from Joy When the uid number is missing the message have is uid in dovecot-uidlist file but is represented in different format: 92 G1448960589.M630964P41853.qb-dev :1448960589.M630964P41853.qb-dev,S=35341,W=35856 93 :1448960586.M423202P41802.qb-dev,S=35378,W=35890 It seems that the uid is determined after the log is written ... I'm running Dovecot 2.2.19, Maildir as storage format and LMTP for delivery. Is a bug or a "feature"? Can be fixed? Thanks this is my configuration: # dovecot -n # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 # OS: Linux 2.6.32-573.7.1.el6.x86_64 x86_64 CentOS release 6.7 (Final) auth_master_user_separator = * dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 89 first_valid_uid = 89 last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes log_path = /var/log/dovecot/dovecot.log mail_fsync = always mail_location = maildir:~/Maildir mail_plugins = quota zlib fts fts_lucene acl mail_log notify managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-deny-sql.conf.ext deny = yes driver = sql } passdb { args = /etc/dovecot/extra/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 fts = lucene fts_lucene = whitespace_chars=@. mail_log_events = copy save mail_log_fields = uid box msgid from subject quota = dict:User quota::noenforcing:proxy::sqlquota sieve_default = /etc/dovecot/sieve/default.sieve zlib_save = gz zlib_save_level = 6 } protocols = imap sieve lmtp service auth { unix_listener auth-userdb { group = vchkpw mode = 0660 user = vmail } } service dict { unix_listener dict { group = vchkpw mode = 0660 user = vmail } } service imap-postlogin { executable = script-login /etc/dovecot/scripts/imap-postlogin.sh unix_listener imap-postlogin { group = vchkpw mode = 0660 user = vmail } user = vmail } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = Hello! I run Debian jessie and have an issue with the interaction between Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218) IMAP client, when using Maildir: I use Dovecot LMTP for local delivery into Maildir and access these files both through Dovecot IMAP from the Android Gmail IMAP client and also directly through the filesystem by running Mutt locally on the mail server. When a new mail arrives, Dovecot LMTP will add a new file in Maildir/new and also update the Dovecot index cache. If I then open Mutt and just delete this new message, then Mutt will just delete this file from Maildir/new and the Dovecot index cache will be out-of-sync. (For testing, Mutt can be cut out of the equation entirely by instead just running "rm Maildir/new/*" from the command line to mimic what Mutt does when deleting a new message, but I mention the use of Mutt here to show why this is a real use-case for me.) If I after this open the Gmail app and pull down the message index to refresh from the Dovecot IMAP server, it'll get the cached version of the index, in which the new message is still present (but only the headers, no body will be loaded and the app will show no body snippet). Pulling down the message index again to force a second refresh will cause the new/deleted message to disappear again. This seems to be caused by Dovecot not looking at the Maildir/new timestamp to determine whether the index cache is up-to-date, but instead only looking at the Maildir/cur timestamp. If I instead repeat all these testing steps from the beginning, sending a new mail message, deleting it with "rm Maildir/new/*" but then also running "touch Maildir/cur" before opening the Gmail app and doing the refresh, then Dovecot does update the cache and does not send any ghost messages to the IMAP client. It does seem reasonable to me that Mutt doesn't touch Maildir/cur when just deleting new messages and I suspect that it might be a bug in Dovecot that it timestamp of Maildir/new isn't checked and the index cache updated if that timestamp is newer than the last index update done by the Dovecot LMTP service. Does this analysis seem sound? Did I miss anything? (Output of "doveconf -n" attached.) Cheers // Fredrik Roubert -- Forsterstrasse 64 | +41 78 8170377 CH-8044 Z?rich | https://roubert.name/fredrik/ -------------- next part -------------- # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.2 auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@/ auth_username_format = %Ln first_valid_uid = 1000 last_valid_uid = 65533 listen = 5.148.172.24, 2a02:418:1003:7::1 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader vnd.dovecot.duplicate mbox_read_locks = flock mbox_write_locks = flock namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=SHA512-CRYPT /etc/dovecot/passwd-imap driver = passwd-file } passdb { args = scheme=SHA512-CRYPT /etc/dovecot/passwd-smtp driver = passwd-file } plugin { sieve = ~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/spam.sieve sieve_dir = ~/.sieve sieve_extensions = +editheader +vnd.dovecot.duplicate } protocols = " imap lmtp sieve" service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_cert = References: <565C5DCD.1090706@0tten.nl> Message-ID: <565DA494.5090504@0tten.nl> Op 01-12-15 om 08:41 schreef Steffen Kaiser: > On Mon, 30 Nov 2015, Frido Otten wrote: > > > In our dovecot setup we use 3 different config files. One for > > maildropping(sieve), one for dovecot/imap on standard ports with ssl > > config and one for dovecot/imap on standard ports+1 with a different ssl > > certificate. Currently the client using the standard port+1 config wants > > to make use of shared folders, but the thing is that the current > > namespace separator in all configs is '.'. This is conflicting with > > shared folders. > > How does '.' conflict with shared folders? > Sorry for the confusion, I forgot to mention that we have usernames with a '.' in it. The separator and those usernames together with shared folders, won't work. > > Can we simply run one of the dovecot/imap daemons > > configured with a different separator and shared folders without having > > impact on sieve and the current connected users to the standard ports? > > I hate all questions and answers with "simple" or "simply" in it. Sorry, I'll use easily then... ;) > But yes, the IMAP separator does not have no impact on the filesystem > _usually_. But mayhap you have a strange config anyway (with 3 > configs) and the separator does influence the storage. > Double denial there.. ;) Why 3 different configs? It's for historic reasons, and probably they can be combined into one. So if I understand this right, the separator as a '.' and sieve rules like this with a dot in the foldername:.. if allof (header :contains ["Return-path"] ["@dovecot.org"]) { fileinto "Maillist.Dovecot"; } won't be affected with only one of both imap configs having '/' as a separator. Regards, Frido -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: OpenPGP digital signature URL: From darix at opensu.se Tue Dec 1 13:58:55 2015 From: darix at opensu.se (Marcus Rueckert) Date: Tue, 1 Dec 2015 14:58:55 +0100 Subject: Looks like a bug to me: Dovecot ignores Maildir/new timestamp In-Reply-To: <20151201103010.GB13914@sork.roubert.net> References: <20151201103010.GB13914@sork.roubert.net> Message-ID: <20151201135855.GF23963@nordisch.org> On 2015-12-01 11:30:10 +0100, Fredrik Roubert wrote: > I run Debian jessie and have an issue with the interaction between > Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218) > IMAP client, when using Maildir: As a fellow mutt user. I use mutt with dovecot. mutt headercache was much slower for me than dovecot's caching. if you dont want to go via the tcp port, you can set /usr/lib/dovecot/imap as a tunnel in your muttrc. hth darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From tss at iki.fi Tue Dec 1 14:28:51 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Dec 2015 16:28:51 +0200 Subject: Looks like a bug to me: Dovecot ignores Maildir/new timestamp In-Reply-To: <20151201103010.GB13914@sork.roubert.net> References: <20151201103010.GB13914@sork.roubert.net> Message-ID: > On 01 Dec 2015, at 12:30, Fredrik Roubert wrote: > > Hello! > > I run Debian jessie and have an issue with the interaction between > Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218) > IMAP client, when using Maildir: > > I use Dovecot LMTP for local delivery into Maildir and access these > files both through Dovecot IMAP from the Android Gmail IMAP client and > also directly through the filesystem by running Mutt locally on the mail > server. > > When a new mail arrives, Dovecot LMTP will add a new file in Maildir/new > and also update the Dovecot index cache. If I then open Mutt and just > delete this new message, then Mutt will just delete this file from > Maildir/new and the Dovecot index cache will be out-of-sync. > > (For testing, Mutt can be cut out of the equation entirely by instead > just running "rm Maildir/new/*" from the command line to mimic what Mutt > does when deleting a new message, but I mention the use of Mutt here to > show why this is a real use-case for me.) > > If I after this open the Gmail app and pull down the message index to > refresh from the Dovecot IMAP server, it'll get the cached version of > the index, in which the new message is still present (but only the > headers, no body will be loaded and the app will show no body snippet). > Pulling down the message index again to force a second refresh will > cause the new/deleted message to disappear again. > > This seems to be caused by Dovecot not looking at the Maildir/new > timestamp to determine whether the index cache is up-to-date, but > instead only looking at the Maildir/cur timestamp. Dovecot does check Maildir/new timestamp, but it'll only perform a "partial sync" then, which means it adds any newly found mails but it doesn't delete any mails because it can't be sure that they weren't moved to cur/. One simple fix for this would be to scan cur/ also whenever new/ changes, but that would unnecessarily make the performance worse for most people. Another fix would be to remember that the mail was in new/ and when cur's timestamp hadn't changed and the mail wasn't in new/ anymore it would be removed from index. But this would require remembering that the mail is in new/, which isn't currently done and adding it would probably cause extra disk writes. So I don't really see a way of fixing this in any nice way. From tss at iki.fi Tue Dec 1 14:49:51 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Dec 2015 16:49:51 +0200 Subject: Let lmtp create target directories In-Reply-To: <20151126021202.GA30712@fishbowl.rw.madduck.net> References: <20151126021202.GA30712@fishbowl.rw.madduck.net> Message-ID: On 26 Nov 2015, at 04:12, martin f krafft wrote: > > Hello, > > we're using vmm? to manage our postfix+dovecot virtual mail setup, > which allows us to give every virtual user a separate EUID and every > domain a separate EGID for additional security (vs. handling all > virtual mail with a single "vmail" user). > > As a consequence, however, vmm must itself create the user > directories with the appropriate owners, and to do so, it requires > root rights. > > I am trying to investigate getting rid of this need?. Since Dovecot > quite happily creates ~/Maildir when necessary, couldn't it also > create parents? The home directory should be trivial (same > EUID/EGID), but grandparents etc. might need a different policy > (e.g. 0/EGID for the grandparent, 0/0 for great-grandparents, etc.). Dovecot already creates all the parent directories. What to set to the permissions are of course a problem. http://wiki2.dovecot.org/SharedMailboxes/Permissions explains how it works right now. See especially "Permissions to new /domain/user directories" and "Permissions to new user home directories (v2.2+)". > Is this something that could fall within the realm of Dovecot's > lmtp? Or is the lmtp invoked as the user and doesn't actually drop > root? If so, might there be another way? That's the even bigger issue. The home dir creation is done with the user's privileges, not as root. But the +t bit might help. From tss at iki.fi Tue Dec 1 14:51:58 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Dec 2015 16:51:58 +0200 Subject: CentOS rpm dovecot 2.2.10 auth/db-ldap.c TLS bug/patch In-Reply-To: References: Message-ID: On 25 Nov 2015, at 15:42, Andrey Fesenko wrote: > > Hello, > CentOS rpm dovecot 2.2.10 ?ontains bug auth/db-ldap.c TLS (not connect > LDAP+TLS server ldaps://), exist bug/patch > https://bugs.centos.org/view.php?id=8267 > > As far as the correct patch in upstream dovecot quite a lot of changes > at this point if there is a correct patch? http://hg.dovecot.org/dovecot-2.2/rev/727acba74cbf From stephan at rename-it.nl Tue Dec 1 15:04:58 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 1 Dec 2015 16:04:58 +0100 Subject: lmtp panic In-Reply-To: <5641E721.9060008@bgoperator.com> References: <5641E721.9060008@bgoperator.com> Message-ID: <565DB71A.8030702@rename-it.nl> Op 10-11-2015 om 13:46 schreef Sergey Schwartz: > Gents, > > I've just upgraded to the latest build of dovecot , now lmtp delivery > process panics for just one user Hmm, this could very well be a Sieve issue. Can you obtain a GDB backtrace for this problem (http://www.dovecot.org/bugreport.html)? Can you reproduce it using the sieve-test tool with the involved Sieve script and some example e-mail? Regards, Stephan. > > > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Panic: file str.c: line 22 (str_new_const): assertion failed: > (str[len] == '\0') > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Panic: file str.c: line 22 (str_new_const): assertion failed: > (str[len] == '\0') > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) > [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) > [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) > [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) > [0x7f50e380ddbd] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) > [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) > [0x7f50e380f555] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f50e37eb2e7] > -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) > [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) > [0x7f50e37fe6ea] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) > [0x7f50e37ff277] -> > /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) > [0x7f50e3a62b81] -> > /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) > [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> > dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) > [0x7f50e597ebbc] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) > [0x7f50e597ffb1] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7f50e597ec45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f50e597ede8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) > [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) > [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) > [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) > [0x7f887d2b0dbd] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) > [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) > [0x7f887d2b2555] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f887d28e2e7] > -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) > [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) > [0x7f887d2a16ea] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) > [0x7f887d2a2277] -> > /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) > [0x7f887d505b81] -> > /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) > [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> > dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) > [0x7f887f421bbc] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) > [0x7f887f422fb1] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7f887f421c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f887f421de8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] > Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local > Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Panic: file str.c: line 22 (str_new_const): assertion failed: > (str[len] == '\0') > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) > [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) > [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) > [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) > [0x7fd26aef5dbd] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) > [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) > [0x7fd26aef7555] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7fd26aed32e7] > -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) > [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) > [0x7fd26aee66ea] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) > [0x7fd26aee7277] -> > /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) > [0x7fd26b14ab81] -> > /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) > [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> > dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) > [0x7fd26d066bbc] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) > [0x7fd26d067fb1] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7fd26d066c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7fd26d066de8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7494 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7497 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7499 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 6719 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7502 killed with signal 6 (core > dumped) > Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: > Connection closed (in DATA finished) > > > Oleg has lots and lots of small emails :) > > I'm running ubuntu server 14.04 x86_64 > dovecot 2.2.19 (0b1c73b01a5a) build from > http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main > From antonello.cioffi at uniparthenope.it Tue Dec 1 15:10:56 2015 From: antonello.cioffi at uniparthenope.it (Antonello Cioffi) Date: Tue, 01 Dec 2015 16:10:56 +0100 Subject: Dovecot doesn't sent rejection message user overquota Message-ID: <565DB880.7050601@uniparthenope.it> Hi I'm using postfix+dovecot (2.2.18). The problem is that dovecot silently discard message when user has its own mailbox full without sending rejection message to the sender. Here a sample log: Dec 1 14:54:23 posta2 postfix/smtp[21478]: B315111C00B: to=, relay=192.168.241.110[192.168.241.110]:25, delay=0.4, delays=0.2/0/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as C27BF1244DD) Dec 1 14:54:23 posta2 dovecot: lda(antonen): Error: sieve: msgid=<565DA68E.8060308 at uniparthenope.it>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) Dec 1 14:54:23 posta2 dovecot: lda(antonen): sieve: Execution of script /usr/local/etc/dovecot/sieve/default.sieve failed with unsuccessful implicit keep Dec 1 14:54:23 posta2 dovecot: lda(antonen): msgid=<565DA68E.8060308 at uniparthenope.it>: rejected: Quota exceeded (mailbox for user is full) Dec 1 14:54:23 posta2 postfix/pipe[21020]: 111FB11C00B: to=, relay=dovecot, delay=0.14, delays=0.05/0.01/0/0.09, dsn=2.0.0, status=sent (delivered via dovecot service) This is dovecot configuration: posta2:/usr/local/etc/dovecot # doveconf -n # 2.2.18: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 3.0.101-0.46-default x86_64 SUSE Linux Enterprise Server 11 (x86_64) auth_mechanisms = plain login auth_username_format = %Ln auth_verbose = yes default_internal_user = vmail default_login_user = nobody disable_plaintext_auth = no first_valid_uid = 100 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_gid = 100 mail_location = maildir:%h mail_plugins = " quota" mail_uid = 1002 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate spamtest spamtestplus passdb { args = /usr/local/etc/dovecot/dovecot-people-ldap.conf.ext driver = ldap } plugin { last_login_dict = redis:host=127.0.0.1:port=6379 mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/ sieve_dir = ~/.sieve sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric } postmaster_address = postmaster at uniparthenope.it protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 vsz_limit = 256 M } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } service_count = 0 vsz_limit = 256 M } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl_ca = References: <565DB880.7050601@uniparthenope.it> Message-ID: On 01 Dec 2015, at 17:10, Antonello Cioffi wrote: > > Hi > > I'm using postfix+dovecot (2.2.18). > > The problem is that dovecot silently discard message when user has its own mailbox full without sending rejection message to the sender. > > Here a sample log: > > Dec 1 14:54:23 posta2 postfix/smtp[21478]: B315111C00B: to=, relay=192.168.241.110[192.168.241.110]:25, delay=0.4, delays=0.2/0/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as C27BF1244DD) > Dec 1 14:54:23 posta2 dovecot: lda(antonen): Error: sieve: msgid=<565DA68E.8060308 at uniparthenope.it>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) > Dec 1 14:54:23 posta2 dovecot: lda(antonen): sieve: Execution of script /usr/local/etc/dovecot/sieve/default.sieve failed with unsuccessful implicit keep > Dec 1 14:54:23 posta2 dovecot: lda(antonen): msgid=<565DA68E.8060308 at uniparthenope.it>: rejected: Quota exceeded (mailbox for user is full) > Dec 1 14:54:23 posta2 postfix/pipe[21020]: 111FB11C00B: to=, relay=dovecot, delay=0.14, delays=0.05/0.01/0/0.09, dsn=2.0.0, status=sent (delivered via dovecot service) Dovecot should have sent a bounce message at that time. I guess that doesn't work for some reason. By default Dovecot uses: sendmail_path = /usr/sbin/sendmail Also you can instead have it send it via SMTP by setting submission_host setting. And finally if you give -e parameter to dovecot-lda, it won't send a bounce itself but instead will exit with 77 causing Postfix to send the bounce. From tss at iki.fi Tue Dec 1 15:31:36 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Dec 2015 17:31:36 +0200 Subject: mail_log plugin and uid=error in log file In-Reply-To: <565D6A1E.9090405@skye.it> References: <565D6A1E.9090405@skye.it> Message-ID: <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> On 01 Dec 2015, at 11:36, Alessio Cecchi wrote: > > Hi, > > I enabled notify and mail_log plugins to log the message uid for new emails delivered via LMTP. All works fine except that the uid is not always logged (see uid=error instead of uid=NUMBER) > > from dovecot.log: > > Dec 01 10:03:08 lmtp(alessio at domain.com): Info: copy from : box=INBOX, uid=error, msgid=<14489594 at domain.com>, from="WordPress" , subject=Enquiry from Joy > > Dec 01 10:03:09 lmtp(alessio at domain.com): Info: copy from : box=INBOX, uid=92, msgid=<03c4e16d at www.domain.com>, from="WordPress" , subject=Enquiry from Joy Does this help? http://hg.dovecot.org/dovecot-2.2/rev/25d63d9c7f5a From gilles.chauvin at univ-rouen.fr Tue Dec 1 16:08:01 2015 From: gilles.chauvin at univ-rouen.fr (Gilles Chauvin) Date: Tue, 1 Dec 2015 17:08:01 +0100 Subject: mail_log plugin and uid=error in log file In-Reply-To: <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> References: <565D6A1E.9090405@skye.it> <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> Message-ID: <565DC5E1.1070401@univ-rouen.fr> On 01/12/2015 16:31, Timo Sirainen wrote: > > Does this help? http://hg.dovecot.org/dovecot-2.2/rev/25d63d9c7f5a > Hi Timo, Sorry about the thread hijacking but, speaking about the mail_log plugin, what do you think about the ability to add the session number to the log lines produced by this plugin? This could be a useful information to have too, especially on a large traffic mailhost. Regards, Gilles -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3086 bytes Desc: S/MIME Cryptographic Signature URL: From darix at opensu.se Tue Dec 1 16:14:36 2015 From: darix at opensu.se (Marcus Rueckert) Date: Tue, 1 Dec 2015 17:14:36 +0100 Subject: mail_log plugin and uid=error in log file In-Reply-To: <565DC5E1.1070401@univ-rouen.fr> References: <565D6A1E.9090405@skye.it> <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> <565DC5E1.1070401@univ-rouen.fr> Message-ID: <20151201161436.GG23963@nordisch.org> On 2015-12-01 17:08:01 +0100, Gilles Chauvin wrote: > Sorry about the thread hijacking but, speaking about the mail_log plugin, > what do you think about the ability to add the session number to the log > lines produced by this plugin? > > This could be a useful information to have too, especially on a large > traffic mailhost. mail_log_prefix = "%s(%u): %{session} " only downside is some lines will then end up with the session ID twice. When i asked Timo said he doesnt want to change that behavior in 2.2. but in 2.3 it might be an option to unify that. hth darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From larryrtx at gmail.com Tue Dec 1 16:27:24 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Tue, 1 Dec 2015 10:27:24 -0600 Subject: mail_log & doveadm Message-ID: Is there any way to have doveadm NOT log using mail_log, but have the daemons do it? (or have doveadm's log go to syslogd)? Thanks! ex: doveadm(ler): Info: copy from SA/FN: box=#ARCHIVE/2015/11/SA/FN, uid=1, msgid=, size=107835, vsize=110376, from="Alley Boost" , subject=Alley Boost: VIP Mixer TONIGHT, Startup Expo TOMORROW, & More! :), flags=(\Seen) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 -------------- next part -------------- A non-text attachment was scrubbed... Name: dc.n Type: application/octet-stream Size: 3928 bytes Desc: not available URL: From gilles.chauvin at univ-rouen.fr Tue Dec 1 16:29:13 2015 From: gilles.chauvin at univ-rouen.fr (Gilles Chauvin) Date: Tue, 1 Dec 2015 17:29:13 +0100 Subject: mail_log plugin and uid=error in log file In-Reply-To: <20151201161436.GG23963@nordisch.org> References: <565D6A1E.9090405@skye.it> <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> <565DC5E1.1070401@univ-rouen.fr> <20151201161436.GG23963@nordisch.org> Message-ID: <565DCAD9.8000305@univ-rouen.fr> On 01/12/2015 17:14, Marcus Rueckert wrote: > mail_log_prefix = "%s(%u): %{session} " > > only downside is some lines will then end up with the session ID twice. > When i asked Timo said he doesnt want to change that behavior in 2.2. > but in 2.3 it might be an option to unify that. > Hi Marcus, I will try this soon. Thanks for the tip. Regards, Gilles -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3086 bytes Desc: S/MIME Cryptographic Signature URL: From g.danti at assyoma.it Tue Dec 1 18:15:45 2015 From: g.danti at assyoma.it (Gionatan Danti) Date: Tue, 01 Dec 2015 19:15:45 +0100 Subject: Questions about hardlinks, alternate storage and compression] In-Reply-To: References: <20151126141529.GO18514@frodo.gerdesas.com> <20151126150115.2F0BA23488@talvi.dovecot.org> <565C071F.1000302@assyoma.it> <565C6FE2.3080105@assyoma.it> Message-ID: <68a42790196288a933e9fa644091b10d@assyoma.it> Il 30-11-2015 23:23 Timo Sirainen ha scritto: > On 30 Nov 2015, at 17:48, Gionatan Danti wrote: >> >> Hi Timo, >> glad to know it is in your TODO list ;) > > It's been for many years. > >> Any rough ETA on that? > > Right now it doesn't seem likely to be developed anytime soon. > Thank you anyway :) It let me wonder if I am the only one caring about hardlinks... Maybe I overstate the space savings? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From tss at iki.fi Tue Dec 1 19:37:38 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Dec 2015 21:37:38 +0200 Subject: mail_log & doveadm In-Reply-To: References: Message-ID: <140DA3E0-09CB-4A4D-AA8C-8C8499AF3DB5@iki.fi> On 01 Dec 2015, at 18:27, Larry Rosenman wrote: > > Is there any way to have doveadm NOT log using mail_log, but have the > daemons do it? > (or have doveadm's log go to syslogd)? > > Thanks! > > ex: > doveadm(ler): Info: copy from SA/FN: box=#ARCHIVE/2015/11/SA/FN, uid=1, > msgid=, size=107835, vsize=110376, > from="Alley Boost" , subject=Alley Boost: VIP Mixer > TONIGHT, Startup Expo TOMORROW, & More! :), flags=(\Seen) One possibility is to include mail_log plugin for all except the doveadm: protocol !doveadm { mail_plugins = $mail_plugins mail_log } From crohmann at netcologne.de Wed Dec 2 00:52:22 2015 From: crohmann at netcologne.de (Christian Rohmann) Date: Wed, 2 Dec 2015 01:52:22 +0100 (CET) Subject: Logstash pattern (GROK, KV, ...) to parse dovecot logs anyone? Message-ID: <2111481141.5119.1449017542188.JavaMail.open-xchange@cc-app2.netcologne.de> Hello dovecot-users, I am currently playing with Elastics ELK stack and was kind of surprised to NOT yet find a good set of GROK or KV pattern to parse dovecots lush and information rich logs. The last post regarding this endeavor was in 2014 (http://www.dovecot.org/list/dovecot/2014-June/096589.html), which "only" extracts the key->value pairs but not other parts of the log lines. One finds the occasional attempt here and there on GitHub, like https://github.com/PCextreme/logstash-grok-patterns/blob/master/mail . But nothing in comparison to the simply amazingly good patterns there are for Postfix from whyscream (https://github.com/whyscream/postfix-grok-patterns). He even added some "I don't understand this yet" rule to learn where the parsing lags. I was wondering if anyone here is running logstash and does already have a set of GROK or KV configuration and is willing to share that with the world? A joint effort might get us to a complete extraction of key->values and all other interesting fields for dovecot quickly I hope. Regards Christian From antonello.cioffi at uniparthenope.it Wed Dec 2 08:13:04 2015 From: antonello.cioffi at uniparthenope.it (Antonello Cioffi) Date: Wed, 02 Dec 2015 09:13:04 +0100 Subject: Dovecot doesn't sent rejection message user overquota In-Reply-To: References: <565DB880.7050601@uniparthenope.it> Message-ID: <565EA810.7020700@uniparthenope.it> Il 01/12/2015 16:19, Timo Sirainen ha scritto: > On 01 Dec 2015, at 17:10, Antonello Cioffi wrote: >> Hi >> >> I'm using postfix+dovecot (2.2.18). >> >> The problem is that dovecot silently discard message when user has its own mailbox full without sending rejection message to the sender. >> >> Here a sample log: >> >> Dec 1 14:54:23 posta2 postfix/smtp[21478]: B315111C00B: to=, relay=192.168.241.110[192.168.241.110]:25, delay=0.4, delays=0.2/0/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as C27BF1244DD) >> Dec 1 14:54:23 posta2 dovecot: lda(antonen): Error: sieve: msgid=<565DA68E.8060308 at uniparthenope.it>: failed to store into mailbox 'INBOX': Quota exceeded (mailbox for user is full) >> Dec 1 14:54:23 posta2 dovecot: lda(antonen): sieve: Execution of script /usr/local/etc/dovecot/sieve/default.sieve failed with unsuccessful implicit keep >> Dec 1 14:54:23 posta2 dovecot: lda(antonen): msgid=<565DA68E.8060308 at uniparthenope.it>: rejected: Quota exceeded (mailbox for user is full) >> Dec 1 14:54:23 posta2 postfix/pipe[21020]: 111FB11C00B: to=, relay=dovecot, delay=0.14, delays=0.05/0.01/0/0.09, dsn=2.0.0, status=sent (delivered via dovecot service) > Dovecot should have sent a bounce message at that time. I guess that doesn't work for some reason. By default Dovecot uses: > > sendmail_path = /usr/sbin/sendmail > > Also you can instead have it send it via SMTP by setting submission_host setting. And finally if you give -e parameter to dovecot-lda, it won't send a bounce itself but instead will exit with 77 causing Postfix to send the bounce. > Hi I don't set -e parameter. Now I set "submission_host = mail.uniparthenope.it" to use smtp instead of sendmail_path and the result is now: Dec 2 08:58:49 posta2 dovecot: lda(antonen): msgid=<565EA4B9.1020009 at uniparthenope.it>: Permanently failed to send rejection: smtp(mail.uniparthenope.it): DATA failed: 550 5.7.1 no third-party DSNs Regards -- Dott. Antonello Cioffi Ufficio Servizi Informatici Universit? degli Studi di Napoli Parthenope Tel. 081/5475292 - Fax. 081/5475180 From p.heinlein at heinlein-support.de Wed Dec 2 08:25:23 2015 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Wed, 2 Dec 2015 09:25:23 +0100 Subject: doveadm backup -R Message-ID: <565EAAF3.4050601@heinlein-support.de> We're using doveadm -o imapc_user="$USER" -o imapc_password="$PASSWORD" -o imapc_host=$SERVER1 -o pop3c_user="$USER" -o pop3c_password="$PASSWORD" -o pop3c_host=$SERVER1 -D -v backup -R -u "$USER" imapc: to migrate Mails from Courier to Dovecot. We have some complaints on the old system that there are changes on the old system where delete mails are appearing again (IMAP) and/or mails are downloaded several times (POP3). AFAIK there can't be changes on the old system if "backup -R" is used. Or is there any situation where changes on the old system (where we connect with imapc) may happen? Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From tss at iki.fi Wed Dec 2 12:00:02 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 2 Dec 2015 14:00:02 +0200 Subject: doveadm backup -R In-Reply-To: <565EAAF3.4050601@heinlein-support.de> References: <565EAAF3.4050601@heinlein-support.de> Message-ID: <007B3C5E-C4E0-453A-8820-3BB39AB3C151@iki.fi> On 02 Dec 2015, at 10:25, Peer Heinlein wrote: > > > > We're using > > doveadm -o imapc_user="$USER" -o imapc_password="$PASSWORD" -o > imapc_host=$SERVER1 -o pop3c_user="$USER" -o pop3c_password="$PASSWORD" > -o pop3c_host=$SERVER1 -D -v backup -R -u "$USER" imapc: > > to migrate Mails from Courier to Dovecot. > > > We have some complaints on the old system that there are changes on the > old system where delete mails are appearing again (IMAP) and/or mails > are downloaded several times (POP3). > > AFAIK there can't be changes on the old system if "backup -R" is used. > > Or is there any situation where changes on the old system (where we > connect with imapc) may happen? No.. There really shouldn't be any changes to the source when doveadm backup is used. I can't think of any reason for the behavior you're seeing. You could enable pop3c_rawlog_dir and imapc_rawlog_dir though to see what dsync is doing. From darix at opensu.se Wed Dec 2 14:49:02 2015 From: darix at opensu.se (Marcus Rueckert) Date: Wed, 2 Dec 2015 15:49:02 +0100 Subject: Dovecot doesn't sent rejection message user overquota In-Reply-To: <565EA810.7020700@uniparthenope.it> References: <565DB880.7050601@uniparthenope.it> <565EA810.7020700@uniparthenope.it> Message-ID: <20151202144902.GH23963@nordisch.org> On 2015-12-02 09:13:04 +0100, Antonello Cioffi wrote: > Dec 2 08:58:49 posta2 dovecot: lda(antonen): > msgid=<565EA4B9.1020009 at uniparthenope.it>: Permanently failed to send > rejection: smtp(mail.uniparthenope.it): DATA failed: 550 5.7.1 no > third-party DSNs smells like an error message from your smtp server. -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From s.hoogeveen at nederhost.nl Wed Dec 2 15:38:29 2015 From: s.hoogeveen at nederhost.nl (NederHost/Sebastiaan Hoogeveen) Date: Wed, 2 Dec 2015 16:38:29 +0100 Subject: [patch] Fix for bug in TLS/SSL for LMTP with chained certificates Message-ID: Hi, In case of tl;dr: I fixed a bug in TLS support for LMTP which caused chained certificates not to work, and another one which caused certificate read errors to be ignored; the patches are attached to this email. While testing LMTP with TLS and certificate verification by Postfix I discovered that certificate chains are not exchanged properly when using LMTP, even though everything works fine for POP3 and IMAP (both with or without STARTTLS). On LMTP only the server certificate is included in the TLS handshake, no intermediate certificates are provided by the server. The first problem I fixed is that in lib-ssl-iostream/iostream-openssl-context.c errors from the ssl_ctx_use_certificate_chain function are silently ignored because the function returns 0 for a failure but the caller checks for values smaller than 0. This problem is fixed in the tiny patch dovecot-2.2.19-ssl_ctx_certificate_chain_returnvalue.diff. After applying this patch the following error message appears in the logs for LMTP only (IMAP and POP3 still work fine): dovecot: lmtp(20683): Error: SSL context initialization failed, disabling SSL: Can't load SSL certificate: error:0608308E:digital envelope routines:EVP_PKEY_get1_EC_KEY:expecting a ec key It turns out this issue is not related to the reading of the certificate or its associated chain. Somewhere before ssl_ctx_use_certificate_chain is called an error is put in the OpenSSL error queue which is never retrieved. Only after loading the server certificate is the queue checked and because of the previously existing error the chain is not loaded. I think the error is related to setting specific protocol options in ssl_iostream_context_set (which may be different for LMTP than for IMAP or POP3) but I did not investigate this. I made the problem go away by making the following two changes: 1. The ssl_ctx_use_certificate_chain function now empties the OpenSSL error queue before doing its work by calling ERR_get_error() until the queue is empty. 2. The openssl_iostream_error function in a similar fashion empties the queue and returns only the error message for the most recent error (this prevent earlier errors from 'hiding' later/more relevant ones). After applying this second patch LMTP now works properly with certificate chains. Note that this patch makes previously unhandled errors simply 'disappear' from the queue, which may be a Very Bad Thing. I guess there is a more elegant way of handling this "queued error" issue but this works for me now and I'm actually not a C programmer. These two fixes are included in dovecot-2.2.19-lmtp_ssl_bug.diff. I suspect this is the same issue as the one reported by Piotr Rotter to this list on July the 27th. Kind regards, -- Sebastiaan Hoogeveen NederHost https://www.nederhost.nl/ KvK: 34099781 -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.2.19-ssl_ctx_certificate_chain_returnvalue.diff Type: application/octet-stream Size: 683 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-2.2.19-lmtp_ssl_bug.diff Type: application/octet-stream Size: 1028 bytes Desc: not available URL: From alessio at skye.it Wed Dec 2 15:51:03 2015 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 2 Dec 2015 16:51:03 +0100 Subject: mail_log plugin and uid=error in log file In-Reply-To: <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> References: <565D6A1E.9090405@skye.it> <7B40E438-F2B4-4595-98B3-1E5A05B30376@iki.fi> Message-ID: <565F1367.8090305@skye.it> Il 01/12/2015 16:31, Timo Sirainen ha scritto: > On 01 Dec 2015, at 11:36, Alessio Cecchi wrote: >> >> Hi, >> >> I enabled notify and mail_log plugins to log the message uid for new emails delivered via LMTP. All works fine except that the uid is not always logged (see uid=error instead of uid=NUMBER) >> >> from dovecot.log: >> >> Dec 01 10:03:08 lmtp(alessio at domain.com): Info: copy from : box=INBOX, uid=error, msgid=<14489594 at domain.com>, from="WordPress" , subject=Enquiry from Joy >> >> Dec 01 10:03:09 lmtp(alessio at domain.com): Info: copy from : box=INBOX, uid=92, msgid=<03c4e16d at www.domain.com>, from="WordPress" , subject=Enquiry from Joy > > Does this help? http://hg.dovecot.org/dovecot-2.2/rev/25d63d9c7f5a > I applied the patch and seems works fine. Thanks! -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice From mfoley at ohprs.org Wed Dec 2 18:39:55 2015 From: mfoley at ohprs.org (Mark Foley) Date: Wed, 02 Dec 2015 13:39:55 -0500 Subject: Interpreting keywords Message-ID: <201512021839.tB2IdtxB027967@mail.hprs.local> I've marked several messages in Thunderbird using tags. Tags used are: 0 Important 1 Work 2 To Do 3 Personal 4 Later The messages so tagged appear to have the flag fields set in the IMAP Maildir: cur/1449002162.8993_0.mail:2,Sb cur/1449001929.28087_0.mail:2,Sad I've looked in dovecot-keywords and find: $ more dovecot-keywords 0 $label1 1 $label2 2 $label3 3 $label4 I assume these "$label" values are macros that possibly refer to "Important", "Work", etc., but where are these $label's defined? Are they defined in the dovecot configs somewhere or does the mail client just "know" what these correspond to? --Mark From skdovecot at smail.inf.fh-brs.de Thu Dec 3 07:30:52 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 3 Dec 2015 08:30:52 +0100 (CET) Subject: Interpreting keywords In-Reply-To: <201512021839.tB2IdtxB027967@mail.hprs.local> References: <201512021839.tB2IdtxB027967@mail.hprs.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 2 Dec 2015, Mark Foley wrote: > I've marked several messages in Thunderbird using tags. Tags used are: > > 0 Important > > The messages so tagged appear to have the flag fields set in the IMAP Maildir: > > cur/1449002162.8993_0.mail:2,Sb > cur/1449001929.28087_0.mail:2,Sad > > I've looked in dovecot-keywords and find: > > $ more dovecot-keywords > 0 $label1 > > I assume these "$label" values are macros that possibly refer to "Important", "Work", etc., but > where are these $label's defined? Are they defined in the dovecot configs somewhere or does the > mail client just "know" what these correspond to? The latter, see http://superuser.com/questions/983401/import-export-or-retrieve-thunderbird-tags-from-imap-server - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVl/vrHz1H7kL/d9rAQIuhAgApkilRQwWk4wdskpnqpfM9/4hshiTke3P n/3z4wK8J6El2PxSI+pIxdRfIB5wHpa8pJ/Q9LNrWWKUzHQMsTvzdBTbwuf45hj2 KZN9sv1I+3CI3fcBMUkagxI/ds8IK3tNm079HajW7cIFkKRhT8GDKdGsPNJU/WG5 PI7LPctefYWyb8bSEY80946pkAr/dnIybwpS+B6QX9KnHIIktYGcNUIwqZLX1zaR SBiu2rBrrNPNPEQLbyCS+suUoC10+0o+SopP3HmYJjQIjqzh2vtXUTt1m12lgffC GR43JrKVHMTo/ZLM30XQdjNfySLsAOvh/sW/8rWS6dMlBT2aXe/2Fw== =4du1 -----END PGP SIGNATURE----- From f0andrey at gmail.com Thu Dec 3 09:03:51 2015 From: f0andrey at gmail.com (Andrey Fesenko) Date: Thu, 3 Dec 2015 12:03:51 +0300 Subject: CentOS rpm dovecot 2.2.10 auth/db-ldap.c TLS bug/patch In-Reply-To: References: Message-ID: On Tue, Dec 1, 2015 at 5:51 PM, Timo Sirainen wrote: > On 25 Nov 2015, at 15:42, Andrey Fesenko wrote: >> >> Hello, >> CentOS rpm dovecot 2.2.10 ?ontains bug auth/db-ldap.c TLS (not connect >> LDAP+TLS server ldaps://), exist bug/patch >> https://bugs.centos.org/view.php?id=8267 >> >> As far as the correct patch in upstream dovecot quite a lot of changes >> at this point if there is a correct patch? > > http://hg.dovecot.org/dovecot-2.2/rev/727acba74cbf > Thanks this work too, It needs some work though, since the original version are different (hg and rpm dovecot source version) From antonello.cioffi at uniparthenope.it Thu Dec 3 09:55:48 2015 From: antonello.cioffi at uniparthenope.it (Antonello Cioffi) Date: Thu, 03 Dec 2015 10:55:48 +0100 Subject: Dovecot doesn't sent rejection message user overquota In-Reply-To: <20151202144902.GH23963@nordisch.org> References: <565DB880.7050601@uniparthenope.it> <565EA810.7020700@uniparthenope.it> <20151202144902.GH23963@nordisch.org> Message-ID: <566011A4.4060504@uniparthenope.it> Il 02/12/2015 15:49, Marcus Rueckert ha scritto: > On 2015-12-02 09:13:04 +0100, Antonello Cioffi wrote: >> Dec 2 08:58:49 posta2 dovecot: lda(antonen): >> msgid=<565EA4B9.1020009 at uniparthenope.it>: Permanently failed to send >> rejection: smtp(mail.uniparthenope.it): DATA failed: 550 5.7.1 no >> third-party DSNs > smells like an error message from your smtp server. > > I solved: it's a very very old header_checks rule in postfix configuration that I use some times ago to stop some kind of spam and i've forgive it Dec 3 10:28:19 posta2 postfix/cleanup[31760]: 3CC8611C049: reject: header Content-Type: multipart/report; report-type=delivery-status;??boundary="32191/mail.uniparthenope.it" from localhost[127.0.0.1]; from=<> to= proto=ESMTP helo=: 5.7.1 no third-party DSNs thanks for your help bye -- Dott. Antonello Cioffi Ufficio Servizi Informatici Universit? degli Studi di Napoli Parthenope Tel. 081/5475292 - Fax. 081/5475180 From tss at iki.fi Thu Dec 3 10:24:56 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 3 Dec 2015 12:24:56 +0200 Subject: [patch] Fix for bug in TLS/SSL for LMTP with chained certificates In-Reply-To: References: Message-ID: <42802069-DDCD-4E41-8524-A37EDD5BDD88@iki.fi> On 02 Dec 2015, at 17:38, NederHost/Sebastiaan Hoogeveen wrote: > > Hi, > > In case of tl;dr: I fixed a bug in TLS support for LMTP which caused chained certificates not to work, and another one which caused certificate read errors to be ignored; the patches are attached to this email. > > While testing LMTP with TLS and certificate verification by Postfix I discovered that certificate chains are not exchanged properly when using LMTP, even though everything works fine for POP3 and IMAP (both with or without STARTTLS). On LMTP only the server certificate is included in the TLS handshake, no intermediate certificates are provided by the server. > > The first problem I fixed is that in lib-ssl-iostream/iostream-openssl-context.c errors from the ssl_ctx_use_certificate_chain function are silently ignored because the function returns 0 for a failure but the caller checks for values smaller than 0. This problem is fixed in the tiny patch dovecot-2.2.19-ssl_ctx_certificate_chain_returnvalue.diff. Applied. > After applying this patch the following error message appears in the logs for LMTP only (IMAP and POP3 still work fine): > > dovecot: lmtp(20683): Error: SSL context initialization failed, disabling SSL: Can't load SSL certificate: error:0608308E:digital envelope routines:EVP_PKEY_get1_EC_KEY:expecting a ec key > > It turns out this issue is not related to the reading of the certificate or its associated chain. Somewhere before ssl_ctx_use_certificate_chain is called an error is put in the OpenSSL error queue which is never retrieved. Only after loading the server certificate is the queue checked and because of the previously existing error the chain is not loaded. I think the error is related to setting specific protocol options in ssl_iostream_context_set (which may be different for LMTP than for IMAP or POP3) but I did not investigate this. http://hg.dovecot.org/dovecot-2.2/rev/302c3c7e11f8 should fix it. > I made the problem go away by making the following two changes: > > 1. The ssl_ctx_use_certificate_chain function now empties the OpenSSL error queue before doing its work by calling ERR_get_error() until the queue is empty. > > 2. The openssl_iostream_error function in a similar fashion empties the queue and returns only the error message for the most recent error (this prevent earlier errors from 'hiding' later/more relevant ones). > > After applying this second patch LMTP now works properly with certificate chains. Note that this patch makes previously unhandled errors simply 'disappear' from the queue, which may be a Very Bad Thing. I guess there is a more elegant way of handling this "queued error" issue but this works for me now and I'm actually not a C programmer. These two fixes are included in dovecot-2.2.19-lmtp_ssl_bug.diff. I changed this to work the same in lib-ssl-iostream as it works in login-common/ssl-proxy-openssl.c (I wonder why it didn't originally work the same way..) and also merged more of the error handling code in login-common and lib-ssl-iostream. From serbr at runbox.com Thu Dec 3 11:34:21 2015 From: serbr at runbox.com (sb) Date: Thu, 3 Dec 2015 12:34:21 +0100 Subject: How do we disable LOGIN-REFERRALS? Message-ID: <566028BD.4030908@runbox.com> > Network Working Group M. Gahrns > Request for Comments: 2221 Microsoft > Category: Standards Track October 1997 > > IMAP4 Login Referrals ... > 6. Security Considerations > > The IMAP4 login referral mechanism makes use of IMAP URLs, and as > such, have the same security considerations as general internet URLs > [RFC-1738], and in particular IMAP URLs [IMAP-URL]. > > A server MUST NOT give a login referral if authentication for that > user fails. This is to avoid revealing information about the user's > account to an unauthorized user. > > With the LOGIN-REFERRALS capability, it is potentially easier to > write a rogue 'password catching' server that collects login data and > then refers the client to their actual IMAP4 server. Although > referrals reduce the effort to write such a server, the referral > response makes detection of the intrusion easier. From serbr at runbox.com Thu Dec 3 11:46:44 2015 From: serbr at runbox.com (sb) Date: Thu, 3 Dec 2015 12:46:44 +0100 Subject: How do we disable LOGIN-REFERRALS? (part 2) In-Reply-To: <566028BD.4030908@runbox.com> References: <566028BD.4030908@runbox.com> Message-ID: <56602BA4.3030301@runbox.com> >From /opt/src/dovecot-2.2.19/doc/wiki/PasswordDatabase.ExtraFields.Host.txt > Login referrals are an IMAP extension specified by RFC 2221 > [http://www.apps.ietf.org/rfc/rfc2221.html]. They're not supported by many > clients, so you probably don't want to use them normally. Right. > The following clients are known to support login referrals: > > * Pine > * Outlook (but not Outlook Express) We use neither. > Login referrals are used only if the proxy field isn't set. We want neither LOGIN-REFERRALS nor proxy. Dovecot's configure includes the following by default: > capability_banner="IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID > ENABLE IDLE" If the extension is simply hidden from the banner, an attacker could still use the extension. If one removes the string from the banner above, one merely hides the extension name in the banner, or also disables the extension's engine? From tss at iki.fi Thu Dec 3 12:32:50 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 03 Dec 2015 14:32:50 +0200 Subject: How do we disable LOGIN-REFERRALS? (part 2) In-Reply-To: <56602BA4.3030301@runbox.com> References: <566028BD.4030908@runbox.com> <56602BA4.3030301@runbox.com> Message-ID: <56603672.7080007@iki.fi> On 12/03/2015 01:46 PM, sb wrote: > From /opt/src/dovecot-2.2.19/doc/wiki/PasswordDatabase.ExtraFields.Host.txt >> Login referrals are an IMAP extension specified by RFC 2221 >> [http://www.apps.ietf.org/rfc/rfc2221.html]. They're not supported by >> many >> clients, so you probably don't want to use them normally. > Right. >> The following clients are known to support login referrals: >> >> * Pine >> * Outlook (but not Outlook Express) > We use neither. >> Login referrals are used only if the proxy field isn't set. > We want neither LOGIN-REFERRALS nor proxy. > > Dovecot's configure includes the following by default: >> capability_banner="IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID >> ENABLE IDLE" > If the extension is simply hidden from the banner, an attacker could > still use the extension. If the connection is SSL/TLS encrypted, the attacker can't add/modify login referrals. If it's not encrypted, the attacker could just as well insert the LOGIN-REFERRALS to the CAPABILITY reply if it didn't exist there. > If one removes the string from the banner above, one merely hides the > extension name > in the banner, or also disables the extension's engine? As long as Dovecot doesn't return any login-referrals (which it doesn't by default), I don't see why having LOGIN-REFERRALS in the CAPABILITY reply would matter. From serbr at runbox.com Thu Dec 3 13:17:39 2015 From: serbr at runbox.com (sb) Date: Thu, 3 Dec 2015 14:17:39 +0100 Subject: How do we disable LOGIN-REFERRALS? (part 2) In-Reply-To: <56603672.7080007@iki.fi> References: <566028BD.4030908@runbox.com> <56602BA4.3030301@runbox.com> <56603672.7080007@iki.fi> Message-ID: <566040F3.1000900@runbox.com> On 12/3/15 1:32 PM, Timo Sirainen wrote: > As long as Dovecot doesn't return any login-referrals (which it doesn't > by default), I don't see why having LOGIN-REFERRALS in the CAPABILITY > reply would matter. Because a compatible client will use the capability as advertised by the server, and then fail because the server is not honouring its promise. One can hide the capability in the banner, but also wants to disable its engine. You say that dovecot has it disabled by default, but I have no evidence of it, yet. From tss at iki.fi Thu Dec 3 13:49:58 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 3 Dec 2015 15:49:58 +0200 Subject: How do we disable LOGIN-REFERRALS? (part 2) In-Reply-To: <566040F3.1000900@runbox.com> References: <566028BD.4030908@runbox.com> <56602BA4.3030301@runbox.com> <56603672.7080007@iki.fi> <566040F3.1000900@runbox.com> Message-ID: On 03 Dec 2015, at 15:17, sb wrote: > > On 12/3/15 1:32 PM, Timo Sirainen wrote: >> As long as Dovecot doesn't return any login-referrals (which it doesn't >> by default), I don't see why having LOGIN-REFERRALS in the CAPABILITY >> reply would matter. > Because a compatible client will use the capability as advertised by the server, > and then fail because the server is not honouring its promise. > > One can hide the capability in the banner, but also wants to disable its engine. > You say that dovecot has it disabled by default, but I have no evidence of it, yet. I think you need to read how LOGIN-REFERRALs actually work. There is no code that can be disabled on Dovecot side. From tss at iki.fi Thu Dec 3 13:51:54 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 3 Dec 2015 15:51:54 +0200 Subject: v2.2.20 release candidate released Message-ID: http://dovecot.org/releases/2.2/rc/dovecot-2.2.20.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.20.rc1.tar.gz.sig v2.2.20 probably will be released tomorrow or maybe during weekend. + Added mailbox { autoexpunge=