- dovecot - dovecot.org

'director_tag' field returned from passdb lookup results in 'unknown passdb extra field'
by Dan 25 Sep '19

25 Sep '19

Hello all, I'm working with director in tandem with 'director_tags' to group some backend hosts. I've assigned the groups to the hosts by appending a '@' character after the IP address of a backend host under director_mail_servers: ... director_mail_servers = 192.168.0.1@foo 192.168.0.2@bar 192.168.0.3@foobar ... I can confirm that the backend hosts possess the correct tags by issuing: # doveadm director status mail server ip tag vhosts state state changed users 192.168.0.1 foo 100 up - 0 192.168.0.2 bar 100 up - 0 192.168.0.3 foobar 100 up - 0 And examining that the tags have been assigned correctly. My passdb lookup is using mysql, and it appears as such: passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } This file looks like: driver = mysql connect = host=localhost dbname=xxx user=xxx password=xxx password_query = select director_tag, username, domain, password, 'Y' as `proxy` from users where username = '%n' and domain = '%d' iterate_query = select director_tag, username, domain from users I have a database that contains the following columns: username, domain, password, home, UID, GID, active, and director_tag To assign users to a backend, I went off of the documentation that states: > " A passdb lookup can return "director_tag" field containing the wanted tag name. " To fulfill this, I return the 'director_tag' in the passdb lookup, through password_query. The mapping seems to work. When I change the row in the database, the user gets proxied to the new backend on the next connection. However, My question is, Why does this appear in the logs? "dovecot[1963]: imap-login: Debug: Ignoring unknown passdb extra field: director_tag" How could this be an unknown passdb extra field, when the documentation states that you may return a field containing this key? In the file core/src/login-common/client-common-auth.c - Starting at line 142, there is no if condition checking for 'director_tag'. This causes that line to be output to debug handle. Is this intentional? Thank you for your time. Full configuration: auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = yes director_mail_servers = 192.168.0.1@foo 192.168.0.2@bar 192.168.0.3@foobar director_servers = x.x.x.x y.y.y.y director_user_expire = 5 mins disable_plaintext_auth = no doveadm_port = 24245 haproxy_trusted_networks = x.x.x.x y.y.y.y 127.0.0.1 login_greeting = Dovecot At Your Service managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = imap pop3 service director { fifo_listener login/proxy-notify { mode = 0666 user = $default_login_user } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { haproxy = yes port = 143 } inet_listener imaps { haproxy = yes port = 993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director inet_listener pop3 { haproxy = yes port = 110 } inet_listener pop3s { haproxy = yes port = 995 ssl = yes } } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol doveadm { auth_socket_path = director-userdb }

2 1

Can't Get SASL to Work
by Christopher Wooley 25 Sep '19

25 Sep '19

I have been fighting with this for almost 2 days. I can't seem to get the SMTP server to authenticate through SASL. I have turned so many things off and on, I don't remember what I've done, anymore. Can someone look at these and tell me what I'm missing. I get this in the log, anytime postfix tries to authenticate. Sep 24 19:48:14 ubuntu postfix/smtpd[14464]: warning: SASL: Connect to smtpd failed: No such file or directory Sep 24 19:48:14 ubuntu postfix/smtpd[14464]: fatal: no SASL authentication mechanisms Sep 24 19:48:15 ubuntu postfix/master[14048]: warning: process /usr/lib/postfix/sbin/smtpd pid 14464 exit status 1 srw-rw-rw- 1 postfix postfix 0 Sep 24 19:10 auth 10-auth.conf disable_plaintext_auth = no auth_ssl_require_client_cert = no auth_mechanisms = plain !include auth-system.conf.ext -------------- 10-master.conf service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { mode = 0666 user = postfix group = postfix } } service imap { } service pop3 { } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 user = postfix group = postfix } unix_listener auth-userdb { mode = 0666 user = postfix group = postfix } } service auth-worker { } service dict { unix_listener dict { } } -------------- main.cf smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no readme_directory = no compatibility_level = 2 smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination myhostname = ubuntu.local.acr-live.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases mydestination = $myhostname, ubuntu, localhost.localdomain, xyz.com , localhost mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all inet_protocols = all smtpd_sasl_type = dovecot smtpd_sasl_auth_enable = yes smtp_sasl_auth_enable = yes broken_sasl_auth_clients = yes smtpd_tls_security_level = may ---------------------------------- master.cf smtp inet n - y - - smtpd submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_tls_auth_only=yes -o smtpd_reject_unlisted_recipient=no -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth -o smtpd_recipient_restrictions= -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING pickup unix n - y 60 1 pickup cleanup unix n - y - 0 cleanup qmgr unix n - n 300 1 qmgr tlsmgr unix - - y 1000? 1 tlsmgr rewrite unix - - y - - trivial-rewrite bounce unix - - y - 0 bounce defer unix - - y - 0 bounce trace unix - - y - 0 bounce verify unix - - y - 1 verify flush unix n - y 1000? 0 flush proxymap unix - - n - - proxymap proxywrite unix - - n - 1 proxymap smtp unix - - y - - smtp relay unix - - y - - smtp -o syslog_name=postfix/$service_name showq unix n - y - - showq error unix - - y - - error retry unix - - y - - error discard unix - - y - - discard local unix - n n - - local virtual unix - n n - - virtual lmtp unix - - y - - lmtp anvil unix - - y - 1 anvil scache unix - - y - 1 scache maildrop unix - n n - - pipe flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient} uucp unix - n n - - pipe flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient) ifmail unix - n n - - pipe flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient) bsmtp unix - n n - - pipe flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient scalemail-backend unix - n n - 2 pipe flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension} mailman unix - n n - - pipe flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py ${nexthop} ${user}

2 2

Mail messages are occasionally getting "stuck" behind one message on the server with POP3
by David Salisbury 24 Sep '19

24 Sep '19

I'm currently using Dovecot 2.2.33.2, although I've had this particular issue happen periodically over the last few years with other Dovecot versions. It seems to only affect accounts I'm receiving with POP3, not IMAP ones. What happens is the "top-most" (or, oldest) message in the "cur" folder seems to get "stuck". So I'll, say, come back in after a break to check my email (Thunderbird in my case, latest version), and in one of my folders I'll have 30 copies of the same email (all with the same timestamp), and no newer messages delivered since the timestamp of that message. If I go look in the "cur" folder, it has lots of messages in it. The "new" folder is empty. Every time I click "Get Messages" in the mail client, I get another new copy of the same email and no other mail. If I go manually into the cur folder and move the top-most (again, meaning "oldest") message out (which of course corresponds to the message that I'm receiving copies of over-and-over repeatedly) then when I click "Get Messages" all of the other messages come pouring into my mail client just fine and everything becomes "un-stuck". I've done some experiments, and if I copy that same "bad" message back in to cur with its original filename, it gets stuck again. It is an encrypted message, so I tried decrypting it and putting it back in cur with the original name, and it gets stuck in the same fashion even when it is decrypted. I tried putting the bad message into an account that receives with IMAP, just to see if it would maybe keep messages from delivering or something, but new messages come into the IMAP account just fine, even with the bad message present, so the issue seems specific to POP3. Any ideas what might cause something like that? I've had it happen twice over the last two days (which is unusually frequent, hence my message here), and the message sources are varied. Today it was a normal email message from a person, and yesterday it was a Twitch notification. I don't see anything unusual in the logs (and I did turn on debugging to see if it gave some additional info), just normal POP3 logins and connections closings when I'm clicking "Get Messages". -David

1 0

Any need to be worried about occasional dsync errors ?
by Laura Smith 24 Sep '19

24 Sep '19

I am occasionally (maybe every 4 hours or less frequently) seeing the following two errors appear in my logs. Are they any cause for concern ? Error: Timeout during state=sync_mails (send=done recv=mails) I/O has stalled, no activity for 600 seconds (last sent=mail_request (EOL)

1 0

Quota plug-in, do the directors also need it enabled?
by Kristijan Savic - ratiokontakt GmbH 23 Sep '19

23 Sep '19

I am having an issue with the quota display not showing up in e.g. Roundcube. There are multiple Dovecot servers and three directors. A load balancer also does it's thing. The quota plug-in is enabled everywhere, except on the directors. Do those also need it enabled? -- Kristijan Savic

1 0

pigeonhole sieve "standalone"
by Greg Minshall 22 Sep '19

22 Sep '19

hi. i've been thinking (but, for years now!) of a replacement for procmail, and pigeonhold sieve seems like a good candidate. but, i run nmh, not dovecot. as i started install it (on arch linux), i started to worry that it may be foolish of me to try to run it outside of a dovecot system. any wisdom about that would be gratefully received. thanks in advance, and thanks for the software. Greg Minshall

1 0

mdbox + sis attachments. Error stream is larger than expected by 1 byte
by Oleg Krivonosov 20 Sep '19

20 Sep '19

Hello! I use Dovecot 2.3.4.1-5+deb10u1 (Debian 10 x64) with mdbox, sis and imap. I catch many errors with attachments "Stream is larger than expected" by 1 byte in emails from Xerox WorkCentre 3345. Examples: Sep 20 23:20:04 mailserver dovecot: imap(wasya)<606><AG1zywGTVodVrE84>: Error: Mailbox INBOX: UID=5002: read(attachments-connector(/var/mail/virtual/wasya/mdbox/storage/m.210)) failed: read(/var/mail/attachments/b3/56/b3563977baf186928d38f81480b6443cf6d6d7e4-b86d80062ab36f5d7f190000f7ccf497[base64:18 b/l]) failed: Stream is larger than expected (882801 > 882800, eof=0) (FETCH BODY[]) Sep 20 23:32:27 mailserver dovecot: imap(wasya)<4831><n0a39wGTZ8hVyu8l>: Error: Mailbox INBOX: UID=654996: read(attachments-connector(/var/mail/virtual/wasya/mdbox/storage/m.4372)) failed: read(/var/mail/virtual/wasya/mdbox/storage/m.4372 trailer) failed: Stream is larger than expected (38 > 37, eof=0) (FETCH BODY[]). This error breaks getting email in MS Outlook and breaks viewing attachment in the problem email. Please help me to debug and to fix this issue. -- Best regards Oleg

1 0

Re: Bug] Sieve vacation :addresses match only,>, case-sensitive?
by Klaus Steinberger 20 Sep '19

20 Sep '19

Hi, >> I never saw any mail system in which the local Part ist Case sensitive! > > Every Unix/Linux system (I?d say that represents the majority of mail systems) has a case sensitive local part. Most mail admins have set their systems up to normalize all users on lowercase names, but that is not and has not always been the case. at least in every large installation LDAP User Backend's (or AD or similar) are used, and they are definitly case insensitive for the CN or UID attributes. Really nobody who cares about his systems and users uses mailboxnames which just differ in case (too much danger of confusion!) At least sieve must reflect the same settings as the MDA, so if the MDA is set to downcase (which in my opionion is the only valuable setting) sieve must also downcase. Sincerly, Klaus

1 0

fts_solr: Error: fts_solr: received invalid uid '0'
by Fabian 20 Sep '19

20 Sep '19

Hi, we are trying to add full text search functionality with Solr to our Doveoct setup. Our Versions: OS: Debian 9 Tried versions: - Dovecot 2.2.7 with Solr 3.6 - Dovecot 2.3.4 with Solr 8.2 (2.2.7 from offical Debian repository, 2.3.4 from backports) Search is working mostly of the time perfrectly smooth. But sometimes following message appears in mail.err: dovecot: imap(username)<16189><UxYWLVuSYMasEQoK>: Error: fts_solr: received invalid uid '0' If this error occurs our webmail frontend delivers most of the time a timeout. Sometimes the search only takes really long. Are there any ideas why this error occurs? We are not able to reproduce the error in such a way that it would always be reproducible. However, we can reproduce the behavior in some form over and over again - but we do not know exactly what is decisive. If you need any further information please ask - thanks! :-) dovecont -n: 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.9.0-11-amd64 x86_64 Debian 9.11 ext4 # Hostname: localhost auth_master_user_separator = * auth_username_format = %Ln default_client_limit = 5000 default_process_limit = 1000 default_vsz_limit = 256M dict { sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_uid = 480 imap_capability = +XDOVECOT lmtp_save_to_detail_mailbox = yes mail_attribute_dict = file:/var/spool/dovecot/user/%n/Maildir/dovecot-attributes mail_gid = vmail mail_location = maildir:/var/spool/dovecot/user/%d/%n/Maildir:LAYOUT=fs mail_plugins = " listescape mail_log notify quota acl fts fts_solr virtual" mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { hidden = yes inbox = yes list = no location = prefix = separator = / subscriptions = yes } namespace { list = yes location = maildir:%%h/Maildir:INDEXPVT=~/shared/%%u:LAYOUT=fs prefix = shared/%%n/ separator = / subscriptions = no type = shared } namespace Public { list = children location = maildir:/var/spool/dovecot/public:INDEXPVT=~/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = no type = public } namespace Virtual { hidden = no list = yes location = virtual:/etc/dovecot/virtual:INDEX=/var/spool/dovecot/user/%u/virtual prefix = Virtual/ separator = / subscriptions = no } namespace inbox { inbox = no location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / subscriptions = no type = private } passdb { args = /etc/dovecot/masteruser.%s driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_anyone = allow acl_shared_dict = file:/var/spool/dovecot/public/shared-mailboxes.db fts = solr fts_autoindex = yes fts_solr = url=<a href="http://172.17.10.12:8983/solr/dovecot/" rel="noreferrer noopener" target="_blank">172.17.10.12:8983/solr/dovecot/</a> quota = dict:User quota::no-unset:proxy::sqlquota quota_rule = *:storage=10G sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/spool/dovecot/globalsieve/duplicate.sieve sieve_duplicate_default_period = 1h sieve_duplicate_max_period = 1d } protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service decode2text { executable = script /usr/lib/dovecot/<a href="http://decode2text.sh" rel="noreferrer noopener" target="_blank">decode2text.sh</a> unix_listener decode2text { mode = 0666 } user = dovecot } service dict { unix_listener dict { group = vmail user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = no userdb { args = /etc/dovecot/mysql-userdb.conf.ext driver = sql } protocol lmtp { mail_plugins = " listescape mail_log notify quota acl fts fts_solr virtual acl quota sieve notify" quota_full_tempfail = yes } protocol imap { imap_capability = +XDOVECOT mail_max_userip_connections = 100 mail_plugins = " listescape mail_log notify quota acl fts fts_solr virtual imap_acl imap_quota" } remote <a href="http://172.17.10.10" rel="noreferrer noopener" target="_blank">172.17.10.10</a> { protocol imap { imap_metadata = yes } }

2 3

Second private namespace
by André Rodier 19 Sep '19

19 Sep '19

Hello, I have Dovecot working, with a private namespace only. I would like to know if the following is feasible. - Add a second private namespace, in another directory scheme, for instance /mnt/backup/homebox/<user>/Mails/... - This namespace would be read-only. Perhaps the index and flags should be stored in a different folder with r/w access (e.g. /var, or even /tmp, it's not important) Now, as a matter of fact, this folder would be mounted on demand, via systemd-automount and fuse and is likely to be remote. This might implies some flags or caching on Dovecot performance. Thanks for your insights, André

1 0

Recent Dovecot on old operating system
by geho781＠justmail.de 19 Sep '19

19 Sep '19

Hi, sorry for the dumb question and please ignore this post if you think it's far beyond; i know it's not the way to go, but for reasons ... Has anyone running a self compiled recent dovecot (2.2.36.4) on Debian-7 and does it work? Or thinks it should work. Surprisingly it actually compiles flawlessly on Debian-7, but i wonder wether it will become a complete mess replacingthe existing dovecot (2.1.7) with the new one, serving a few dozens users and any kind of mail clients. Libraries of the compiled Dovecot: $ ldd ./src/master/.libs/dovecot linux-vdso.so.1 => (0x00007ffeacfb1000) libdovecot.so.0 => /usr/local/00-DBAI/dovecot/lib/dovecot/libdovecot.so.0 (0x00007f557b2a9000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f557af1c000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f557ad18000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f557ab10000) /lib64/ld-linux-x86-64.so.2 (0x00007f557b5dc000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f557a8f4000) libc6 versions: Debian-7: 2.13-38 Debian-9: 2.24-11 thanks so far, gerald

2 1

Dovecot 2.3.7.2 lucene segfault and coredump
by Domenico Pastore 19 Sep '19

19 Sep '19

Hi, i have a problem with libclucene-core.so.2.3.3.4. I See segfault error to messages log, when dovecot store mail to INBOX. set 12 13:41:21 box02.par-tec.it kernel: doveadm-server[20285]: segfault at 38 ip 00007f575b68fc42 sp 00007ffc0272e750 error 4 in libclucene-core.so.2.3.3.4[7f575b569000+188000] set 12 13:41:22 box02.par-tec.it kernel: doveadm-server[18871]: segfault at 38 ip 00007f9bea6d2c42 sp 00007fff47bceae0 error 4 in libclucene-core.so.2.3.3.4[7f9bea5ac000+188000] In the dovecot logs: 2019-09-12 13:45:09 auth: Debug: ldap(test4444client(a)newsvilXXXX,10.0.8.128<mailto:test4444client@newsvilXXXX,10.0.8.128>): Finished userdb lookup 2019-09-12 13:45:09 auth: Debug: userdb out: USER 1 test4444client@newsvilXXXX home=/pec_store/pec_mailbox/pec_mailbox_devel01/4/5/6/7/1/45671_18231 quota_rule=*:bytes=1048576000 mail_plugins+= 2019-09-12 13:45:10 SKENO8Uvel0gbAAA5ZKy4w doveadm(test4444client@newsvilXXXX): Fatal: master: service(doveadm): child 27680 killed with signal 11 (core dumped) 2019-09-12 13:45:10 auth: Debug: master in: USER 1 test4444client@newsvilXXXX service=doveadm lip=192.160.10.3 lport=924 rip=10.0.8.128 rport=36760 2019-09-12 13:45:10 auth: Debug: ldap(test4444client@newsvilXXXX 10.0.8.128): Performing userdb lookup I have attach the coredump of error and dovecot.conf Thanks for support. Br, Domenico ------------------- info: ------------------- # 2.3.7.2 (3c910f64b): /pec/config/dovecot/dovecot-devel01/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-957.27.2.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: box02

2 5

fts-elastic plugin
by Filip Hanes 19 Sep '19

19 Sep '19

Hi all, I have recently worked on fts plugin for ElasticSearch. https://github.com/filiphanes/fts-elastic It is forked from fts-elasticsearch as you can see in PR https://github.com/atkinsj/fts-elasticsearch/pull/21 with following changes (maybe some more) - no duplicate json keys in indexed documents, so compatible with ES 7.x - 1 elastic index for all users with routing by username - _id field similar to fts-solr (uid/box_guid/user) - better json response parsing - no need to use _source - added rawlog_dir to config and some more - renamed all elasticsearch to elastic for shorter names - fixed http_client leaks - fixed some compile warnings I would appreciate if someone with better experience in c and dovecot code would review my code and suggest/push changes. In my opinion fts-elastic is now functionally on level as currently recommended FTS plugin: fts-solr. I would like to implement proper rescan inspired by fts-lucene so it would become the most complete fts plugin for dovecot. -- Filip Hanes

3 2

Imaptest stall
by Marc Roos 18 Sep '19

18 Sep '19

I have been testing with imaptest and getting 'stalls', I tried even building from source and static. Even running it on the same host. Anyone knows what I could doing wrong? [@~]# ./imaptest - append=100,0 logout=0 host=192.168.10.44 port=143 user=test2 pass=xxxxxxxx seed=100 secs=240 clients=1 mbox=64kb.mbox box=INBOX/test Logi Sele Appe 100% 100% 100% 1 1 5 1/ 1 0 0 9 1/ 1 0 0 13 1/ 1 0 0 5 1/ 1

3 7

Namespace overlap
by Daniel Miller 17 Sep '19

17 Sep '19

Given an existing default namespace: namespace inbox { type = private separator = / prefix = location = sdbox:/var/mail/%d/%n/sdbox inbox = yes hidden = no list = yes subscriptions = yes } And mailboxes like: INBOX INBOX/Archives INBOX/Archives/2018 if I then define a new namespace: namespace archives { type = private separator = / prefix = Archives/ location = mdbox:/var/mail/%d/%n/Archives/mdbox subscriptions = no list = children } What will happen to the previous existing mailboxes & mails? Will they simply be "masked" by the new namespace and remain pending other operations? Or would they be moved/deleted? If they remain - is it possible to refer to the old mailboxes via either IMAP or doveadm? -- Daniel

1 0

CVE-2019-11500 and LMTP error
by Gabriele Nencioni 16 Sep '19

16 Sep '19

Hi all, does the dovecot fixed version: 2.3.7.2, 2.2.36.4 (as the CVE-2019-11500 says) fix the LMTP error "Got unexpected reply" as well? The LMTP error "Got unexpected reply" is described here: https://dovecot.org/pipermail/dovecot/2018-August/112562.html https://dovecot.org/pipermail/dovecot/2018-August/112666.html Thanks in advance Regards, -- Gabriele Nencioni

2 1

Repo for Debian-Buster
by slater.ch 16 Sep '19

16 Sep '19

Hi, is there already a release date for Debian Buster packages? kind regards Christian

3 2

corrupted mdbox
by sch tel 16 Sep '19

16 Sep '19

Hello list, i run into another problem. It seems there are some null bytes in the middle of an compressed mdbox, if i look into it with hexdump after run doveadm dump. With mdbox-recover.pl i could recover ~2k messages. What should i do to recover this broken mdbox file and dsync can finish? Sep 06 18:47:28 doveadm: Error: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 Sep 06 18:47:28 doveadm: Error: dsync(server1): read(zlib(/var/spool/mail/<user>/storage/m.770)) failed: read(/var/spool/mail/<user>/storage/m.770) failed: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 (last sent=mail, last recv=mail_request (EOL)) Sep 06 18:47:29 dsync-local(<user>)<+mJrOaCNcl2eZwAA0H466A>: Error: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 Sep 06 18:47:29 dsync-local(<user>)<+mJrOaCNcl2eZwAA0H466A>: Error: dsync(server1): read(zlib(/var/spool/mail/<user>/storage/m.770)) failed: read(/var/spool/mail/<user>/storage/m.770) failed: zlib.read(/var/spool/mail/<user>/storage/m.770): unexpected EOF at 7947710 (last sent=mail, last recv=mail_request (EOL))

2 1

question about Mailcrypt plugin
by Daniel Niewerth 14 Sep '19

14 Sep '19

Hi everybody, I have a question about the Dovecot Mailcrypt Plugin. I'm trying to understand what the security of this plugin is based on. The encryption with private and public key and elliptic curves is basically a good thing. But the keys are in the file system together with the encrypted mails. The passwords for the private keys are in the user database. I guess the passwords have to be in plaintext for it to work. Is that true, or did I misunderstand something? Then the encryption would make no sense at all, right? For what purpose was the plugin developed? Can anyone explain this to me? Best regards Daniel

2 2

New to dovecot admin, question about using LDAP for user-specific values
by Gerben Wierda 13 Sep '19

13 Sep '19

I am new to dovecot administration. I’ve read the Wiki but that hasn’t given me the understanding I need. When I query my LDAP (on macOS) on a value for user ‘gerben’, I can get that: dumbledore:~ gerben$ dscl /LDAPv3/127.0.0.1 -read /users/gerben GeneratedUID GeneratedUID: 780D870E-6B00-478E-AB70-3D3307215A82 I would like to use that value in dovecot settings, e.g. something like user_attrs = \ =mail=maildir://Library/Server/Mail/Data/mail/%{ldap:GeneratedUID} Is this possible and if so what do I exactly need to do to get this working? Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ

2 4

Multiple certificate option
by remo＠Mattei.org 13 Sep '19

13 Sep '19

What is the best way to adopt multiple certs? Thanks.

8 9

keywords lost when moving a message in a folder
by wouldsmina 13 Sep '19

13 Sep '19

Hi, I migrate my dovecot server from Debian 9 to Debian 10 (Dovecot 2.3.4.1), since when I move a message in a folder, keywords systematically disappear. I'm using Thunderbird 60.8. Would an option have escaped me? or is there a bug on this version of dovecot? Dovecot 2.3.4.1 OS : Debian 10.0 x64 FS : Ext4 dovecot -n result : # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-5-amd64 x86_64 Debian 10.0 # Hostname: suzuki.domain.tld auth_cache_size = 1 k auth_failure_delay = 3 secs base_dir = /var/run/dovecot/ default_vsz_limit = 4 G first_valid_uid = 100 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = mail system ready. login_trusted_networks = 172.16.0.0/12 192.168.0.0/16 mail_fsync = always mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_prefetch_count = 20 mail_privileged_group = mail mailbox_idle_check_interval = 1 mins mmap_disable = yes namespace inbox { inbox = yes location = maildir:~/Maildir mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { driver = pam } plugin { acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes sieve = ~/sieve/.dovecot.sieve sieve_after = /etc/dovecot/sieve.after sieve_before = /etc/dovecot/sieve.before sieve_before2 = ~/sieve/.dovecot.vacation.sieve sieve_dir = ~/sieve sieve_editheader_forbid_add = X-Verified sieve_editheader_forbid_delete = X-Verified X-Seen sieve_editheader_max_header_size = 1k sieve_extensions = +editheader sieve_max_script_size = 2M } postmaster_address = postmaster(a)domain.tld protocols = " imap lmtp pop3" service auth-worker { process_min_avail = 6 service_count = 0 } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 4 G } service imap { process_limit = 4096 vsz_limit = 4 G } service pop3 { process_limit = 1024 } ssl_cert = </etc/ssl/certs/suzuki.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 25 } thanks.

2 2

Outlook 2019 and IMAP
by John Regan 12 Sep '19

12 Sep '19

Hi, We've just upgraded a few of our desktops to use Outlook 2019, and the configuration is different from previous versions. There doesn't appear to be any ability to specify a username in addition to the user's email address. The username is different than the username specified in the email address. I had read about setting auth_username_format, but the user login name is "user" while the email address is user.name(a)company.com. I've included below my dovecot -n config. # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 5.1.16-300.fc30.x86_64 x86_64 Fedora release 30 (Thirty) # Hostname: orion.inside.example.com mail_gid = mail mail_home = mail/ mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_max_userip_connections = 500 mail_plugins = " listescape" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational rege x imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 } process_min_avail = 6 service_count = 0 vsz_limit = 512 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = 192.168.1.100 port = 995 } } ssl_cert = </etc/letsencrypt/fullchain.pem ssl_cipher_list = ALL:!LOW:!EXP:!aNULL:!RC4::!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } userdb { driver = passwd }

2 1

Re: Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files)
by Zenaan Harkness 12 Sep '19

12 Sep '19

I am wondering why sieve-filter is so slow compared to gnu sieve. I run mpop (like getmail) to download from a pop3 server to a local mbox file: ~/mail/email-incoming-unsorted This step is very fast. The next step, I throw the email-incoming-unsorted mbox file at a sieve processor, to sort the emails from that mbox, into other mboxes, according to the sieve rules file. Up until a couple days ago I was using Gnu sieve. Gnu sieve balks on emails which have no x-message-id (?? something like this) header field, so after a few years, I finally decided to switch "up" to Dovecot/Pigeonhole's "sieve-filter" command. Using Gnu sieve, this mbox sorting step was even faster than mpop (/ getmail) - and mpop and getmail are really fast (compared with fetchmail), since they pipeline the email downloads. Even with 100s of emails, Gnu sieve would take only 10 to 20 seconds at most. Super fast. Using sieve-filter, all emails are being processed - including those without "message id header". This is good. But also, using sieve filter, is really slower - slower than the download step by an order of magnitude or two. See below for details, any ideas appreciated. To add to the below, I added: mbox_very_dirty_syncs = yes to the sieve-filter config, which slightly improves performance, but not by much (in comparison with Gnu sieve). TIA, ----- Forwarded message from Zenaan Harkness <zenaan(a)freedbms.net> ----- From: Zenaan Harkness <zenaan(a)freedbms.net> To: debian-user(a)lists.debian.org Date: Thu, 12 Sep 2019 08:06:12 +1000 Subject: Re: Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files) On Thu, Sep 12, 2019 at 07:55:23AM +1000, Zenaan Harkness wrote: > Why is Gnu sieve so extremely fast to batch process an mbox file, but > while Dovecot's sieve-filter is an order of magnitude slower? > > Sequence: > > - mpop or getmail to pipeline download emails into temp mbox file > - filter that file > > Gnu sieve just flies through a local mbox file and saving emails to > other local mbox files. > > Gnu sieve rejects too many emails with "malformed" errors, so after a > few years I bit the bullet and upgraded to Dovecot's sieve-filter. > > Dovecot's sieve-filter, at present, is an order of magnitude slower. > > Here's my filter command (one line): > > /usr/bin/sieve-filter -veW -c $HOME/etc/email/sieve-dovecot-config.conf -o mail_location=mbox:~/mail:INBOX=~/mail/Inbox:INDEX=:UTF-8:VOLATILEDIR=/tmp/dovecot-volatile/%2.256Nu/%u:SUBSCRIPTIONS=dovecot_subscriptions ~/etc/email/sieve.rc email-incoming-unsorted > > The sieve script is fine now that I have the correct "require" > clauses (hint: "capability strings"). > > File ~/etc/email/sieve-dovecot-config.conf: > > protocols = pop > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > mail_fsync = never > > There's no re-sending of emails into my local Postfix SMTP server - I > checked the system logs and confirmed this (journalctl -f). > > I suspect that Gnu sieve was directly writing each email to the > appropriate sieve-determined mbox file (perhaps with only a sync at > the end of a single batch process - what I've attempted to achieve > above with sieve-filter), and that sieve-filter is instead passing > each email through some (dovecot) lda? > > Here's the output for a sieve-filter batch processing of 11 emails: > > $ /usr/bin/sieve-filter -veW -c /home/zen/etc/email/sieve-dovecot-config.conf -o mail_location=mbox:/home/zen/mail:INBOX=/home/zen/mail/Inbox:INDEX=:UTF-8:VOLATILEDIR=/tmp/dovecot-volatile/%2.256Nu/%u:SUBSCRIPTIONS=dovecot_subscriptions /home/zen/etc/email/sieve.rc email-incoming-unsorted > # PS0 Timestamp: 20190912@07:02:23 > info: filtering: [Tue, 3 Sep 2019 05:17:16 -0500; 10240 bytes] `Re: VentureBeat: The death of disk? H...'. > info: msgid=<CAMjeLr91T9R7APsuxQVuM3WbqDsxAfwn4=OYDeDX4FMcoRdGdQ(a)mail.gmail.com>: stored mail into mailbox 'l/cp/cp'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 07:29:53 -0400; 12968 bytes] `[zfs-devel] xattr naming format in Zo...'. > info: msgid=<15675101930.d5ba2E.12322(a)composer.zfsonlinux.topicbox.com>: stored mail into mailbox 'l/z/zdev'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 03 Sep 2019 15:29:09 +0300; 20461 bytes] `Re: [zfs-devel] xattr naming format i...'. > info: msgid=<23955051567513749(a)sas1-02732547ccc0.qloud-c.yandex.net>: stored mail into mailbox 'l/z/zdev'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 18:20:42 +0530; 18065 bytes] `Re: [Gluster-users] Issues with Geo-r...'. > info: msgid=<CADmkyZMxrfOANrAP+_URAHJcMqCqh=iGdajTSzkfQ5PCZsUfyg(a)mail.gmail.com>: stored mail into mailbox 'l/gl/user'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 09:34:20 -0400; 13342 bytes] `Re: tasksel'. > info: msgid=<20190903133420.GS6166(a)eeg.ccf.org>: stored mail into mailbox 'l/deb/user'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 06:56:07 -0700 (PDT); 12390 bytes] `[awx-project] Re: AWX on Kubernetes m...'. > info: msgid=<0715adb7-540f-4cff-9282-e1252c53c2e8(a)googlegroups.com>: stored mail into mailbox 'l/ansible/awx'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 07:01:27 -0700 (PDT); 12220 bytes] `[awx-project] Re: AWX on Kubernetes m...'. > info: msgid=<949b2c17-4254-49f1-83b4-cd54d15aa17d(a)googlegroups.com>: stored mail into mailbox 'l/ansible/awx'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 10:14:58 -0400; 25313 bytes] `Re: [zfs-devel] xattr naming format i...'. > info: msgid=<CAB5c7xpHCdFx1w3yA9FyRL-KQ8BUiCr4JbiDQRuFJj9nOgKxTg(a)mail.gmail.com>: stored mail into mailbox 'l/z/zdev'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 17:10:22 +0200; 7567 bytes] `Re: [asterisk-users] Playing MP3's in...'. > info: msgid=<20190903151022.354xpe6ds2vglher(a)red.localdomain>: stored mail into mailbox 'l/as/users'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Wed, 4 Sep 2019 01:04:49 +0900; 14858 bytes] `Re: [Hyperledger Fabric] a primitive ...'. > info: msgid=<160901d8-b903-9e9a-91ac-267571b0e24d(a)gmx.com>: stored mail into mailbox 'l/hl/fabric'. > info: message expunged from source mailbox upon successful move. > info: filtering: [Tue, 3 Sep 2019 09:55:22 -0700 (PDT); 13337 bytes] `[awx-project] Re: AWX on Kubernetes m...'. > info: msgid=<f9bc4e6a-8445-4b34-927a-35f577ffcc07(a)googlegroups.com>: stored mail into mailbox 'l/ansible/awx'. > info: message expunged from source mailbox upon successful move. > 2 ▶︎️ zen@eye 20190912@07:02:30 ~ $ > > > So about 3/4 of a second is spent by dovecot's sieve-filter, on each > email that it processes - watching it is painful given how fast Gnu > sieve has been for the last few years - it's almost (but not quite) > as slow as my previous fetchmail email download per-email time. > > Attached is a -D debug run of sieve-filter on 20 emails - slightly > longer than the above, and took roughly 15 seconds to run. > > Any help appreciated... On another test run of ~600 emails, sieve-filter is consistently running ~100% of one CPU (for about 4 minutes) to process these emails, which leads to the conclusion that despite what looks like should be a batch process, sieve-filter is perhaps reloading the rules for every single email that it processes, even though I gave it a whole mbox, and not a single email, to process. Can sieve-filter work the way it should / the way I want it / batch process a whole mbox - without reloading the sieve rules for every email? ----- End forwarded message -----

3 2

Re: [Bug] Sieve vacation :addresses match only,> case-sensitive?
by Klaus Steinberger 12 Sep '19

12 Sep '19

> I want to have it case-insensitive again, like in Pigeonhole version 0.4.24 i would also plea for making it case-insensitive again! We were hit by this too, and I never saw any mail system in which the local Part ist Case sensitive! Sincerly, Klaus Steinberger -- Rechnerbetriebsgruppe / IT, Fakultät für Physik Klaus Steinberger FAX: +49 89 28914280 Tel: +49 89 28914287

6 5

WARNING: using attachment_dir with plugin zlib can corrupt mails
by Patrick Cernko 12 Sep '19

12 Sep '19

Hello list, hello Dovecot developers, this week, I discovered a serious bug in Dovecot, that lead to several broken mails on our servers. The bug corrupts the first few characters of the mail header during saving. On our setup, it was almost always only the very first line of text, that was corrupted. Depending on the IMAP client (they seem to request different header fields, ... during mail access), the bug causes the imap process to hang up the TCP connection and log errors like this: > imap(USERNAME)<4767><TeQP4ASOTK5/AAAB>: Error: Corrupted record in index cache file /IMAP/mail/mailboxes/USERNAME/mdbox/mailboxes/Trash/dbox-Mails/dovecot.index.cache: UID 489113: Broken fields in mailbox Trash: read(attachments-connector(zlib(/IMAP/mail/mailboxes/USERNAME/mdbox/storage/m.813))): FETCH BODY[HEADER.FIELDS (RETURN-PATH SUBJECT)] got too little data: 2 vs 122 In our case that finally grabbed my attention, the client was the users iphone that did not display any new messages but his Thunderbird did. The bug seems to be triggered by a bad "interaction" of attachment_dir option and zlib plugin. If you use both, you most likely are affected, too, except you only use zlib plugin for reading previously compressed stored mails. That's also the workaround we use now: zlib plugin only enabled in mail_plugins but no plugin/zlib_save set. The bug occurs on very specific mails. Due to privacy reasons I could not provide sample mails here. Storing such mails seems to trigger the bug reproducible. I attached a very minimal doveconf -n config, that can be used to trigger the bug. If one of the developers is interested, I can try to generate an "anonymized" version of such a specific mail that still causes the issue. I discovered the bug on our productive systems, running latest Dovecot 2.2 release, but the latest 2.3 I used during debugging is affected, too. During debugging, I also found one hint, that might help find the bug: If you store a problematic mail with zlib_save=gz (or zlib_save=bz2) and then disable the zlib plugin in mail_plugins, you can call doveadm fetch -u test hdr all | grep -v ^hdr: | gzip --decompress on test's mailbox with only that one broken mail. This will display the beginning of the rfc822 mail text until gzip terminates with "gzip: stdin: unexpected end of file", approximately after twice the length of the mail HEADER. This might indicate, that dovecot stores the uncompressed size of the header in it's data structures although the mail is stored compressed. I also found a very efficient way to find all affected mails in our setup: doveadm -f flow fetch -A 'user guid mailbox uid seq flags hdr' all | \ grep -a "^[^ ]+ user=" | \ grep -avF ' hdr=Return-path: ' | \ grep -av '.* hdr=[[:print:][:space:]]*$' (runtime for ~6M mails on our servers was 20-30min) This can be even more optimized if you have a powerful storage system with GNU parallel: > doveadm user '*' | parallel "doveadm -f flow fetch -u '{}' 'user guid mailbox uid seq flags hdr' all | grep -a '^user=' | grep -avF ' hdr=Return-path: ' | grep -av '.* hdr=[[:print:][:space:]]*$' || true" (runtime for ~6M mails on our servers was ~4min) The command will give you a list of mails that possibly are affected, check the full output of doveadm fetch -u USERNAME hdr guid GUID | less to verify that the header is really broken. On our systems I found 39 mails within ~12M mails. I was able to recover these mails "manually" by reconstructing the Return-Path header line, importing the fixed mails and expunging the corrupt ones. Before importing, I had to disable zlib_save option obviously. Best regards, -- Patrick Cernko <pcernko(a)mpi-klsb.mpg.de> +49 681 9325 5815 Joint Administration: Information Services and Technology Max-Planck-Institute fuer Informatik & Softwaresysteme

4 4

pigeonhole/lda accessing -m folder
by matthias lay 12 Sep '19

12 Sep '19

Hi all, anybody knows, if there is a way to acces the folder from a call like this deovecot-lda -m destfolder from within a sieve script? thx in advance matze

5 9

Re: Gnu sieve vs Dovecot sieve-filter - sieve-filter extremely slow at lda (writing emails to local mbox files)
by Zenaan Harkness 12 Sep '19

12 Sep '19

(I did subscribe to this mailing list, albeit with zen at freedbms.net, so either way I'm getting all your emails - thank you -so- much for replying...) MUA is mutt, reading email in a terminal (sorry, forgot to mention this before). For many years now my email folder (mbox files) collection has grown to many GiB, mostly mailing lists. If I am to change email storage format, it should be mutt compatible; looking at https://wiki2.dovecot.org/MailboxFormat I see that only DJB's Maildir is compatible with both Dovecot ("a reliable choice" says the wiki), and mutt. I can imagine that sdbox or mdbox could be made "mutt compatible" so to speak, by running some sort of local IMAP server, and accessing my email from mutt that way; this is undesirable to my mind because this would require: 1) a new learning curve wrt mutt and reading email on IMAP servers 2) a new learning curve to set up a local IMAP server (securely) 3) the inability to use mutt without a local IMAP server to read my local email but such a setup would also have some quite desirable benefits: 1) once set up, multiple MUAs could be used, and I'd have a beginning grasp on setting up an IMAP server and front ends (this is something on my bucket list, to assist my local church with) 2) simpler remote "online" access to my local "offline" email store (e.g. using my mobile phone when on the road) by setting up a webmail server (much simpler (read "possible" to use on a mobile phone) than using a vpn and mutt...), thus freeing me up from the behemoth web email providers... Next, I do not know how to "pipe the messages to the dovecot lda". After downloading from my POP3 provider into a local mbox file (this is my step 1), then I sort the emails (this is my step 2): the following should be on a single line: /usr/bin/sieve-filter -veW -c $HOME/etc/email/sieve-dovecot-config.conf -o mail_location=mbox:~/mail:INBOX=~/mail/Inbox:INDEX=:UTF-8:VOLATILEDIR=/tmp/dovecot-volatile/%2.256Nu/%u:SUBSCRIPTIONS=dovecot_subscriptions ~/etc/email/sieve.rc email-incoming-unsorted As you can see from the above command, sieve-filter is given the name of the mbox ("mail folder") to sort, as its very last argument on the command line - so in this instance, sieve-filter really has no excuse, and should be not be re-reading the sieve rules script for each email - now perhaps that's not happening, I only made an assumption because of a CPU hitting 100% for a minute or two just to process a few 100 emails... What could also be happening (again, an assumption), is that sieve-filter is written to assume dovecot index files to be in existence. I disabled those with the "INDEX=" clause you see in the command above, which obviously has been given no value. The reason I figured out how to disable the creation of the indexes in the .imap directories, is that for my setup, Gnu sieve has proven that I should not need such indexes - with mbox files, just append each email to the end of the target "mailbox folder" mbox file, and we're done! This literally should not cost 100% CPU, even for one millisecond! But more importantly, because my working email folder is ~30GiB, without disabling this index creation step, sieve-filter forced the creation of indexes, which "took so long I gave up and hit CTRL-C, which did not work, so I kill -9'ed the sieve-filter and whatever other process was not stopping". Last year someone on debian-user recommended I upgrade to using Dovecot/Pigeonhole's sieve-filter (rather than Gnu sieve) due to the issues with Gnu sieve. I am starting to think that I should perhaps try to figure out if it's possible to (re)process the emails Gnu sieve has a problem with, to massage them into a shape that Gnu sieve accepts - then my immediate problem would certainly be solved... Thank you all again.. Zenaan

1 1

Subscribe to a fileinto :create mailbox?
by Larry Rosenman 11 Sep '19

11 Sep '19

Is it possible in pigeonhole? I.E. I do a fileinto :create "some/mail/box" How can I make it autosubscribe? doveconf -n: # 2.3.7.2 (3c910f64b): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: FreeBSD 12.0-STABLE amd64 # Hostname: thebighonker.lerctr.org auth_default_realm = lerctr.org auth_mechanisms = plain login auth_realms = lerctr.org thebighonker.lerctr.org tbh.lerctr.org thejonesonair.com thejonesonair.net why.net default_vsz_limit = 1 G deliver_log_format = msgid=%m: %$ (subject=%s from=%f size=%w) doveadm_password = # hidden, use -P to show it first_valid_gid = 0 first_valid_uid = 0 lda_mailbox_autocreate = yes listen = 192.147.25.65, :: login_access_sockets = tcpwrap mail_attribute_dict = file:%h/mail/.imap/dovecot-mail-attributes mail_location = mbox:~/mail:INBOX=~/mail/INBOX mail_log_prefix = "%s(%u/%p): " mail_plugins = " fts fts_solr notify virtual" mail_privileged_group = mail mail_server_admin = mailto:ler@lerctr.org mail_server_comment = LERCTR Mail Server managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader mboxmetadata servermetadata vnd.dovecot.debug imapsieve vnd.dovecot.imapsieve namespace archive { hidden = no list = no location = mbox:~/MAIL-ARCHIVE prefix = ARCHIVE/ separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { auto = create } mailbox SENT { special_use = \Sent } mailbox SPAM { special_use = \Junk } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox virtual/Flagged { special_use = \Flagged } mailbox virtual/all { special_use = \All } prefix = separator = / } namespace virtual { hidden = no list = yes location = virtual:~/MAIL-VIRTUAL prefix = Virtual/ separator = / } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = user=%Ln noauthenticate driver = static skip = authenticated } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam override_fields = domain=lerctr.org skip = authenticated } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_mailbox1_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = SPAM imapsieve_mailbox2_before = file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = SPAM imapsieve_mailbox2_name = * imapsieve_url = sieve://thebighonker.lerctr.org mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject vsize flags recipient_delimiter = +-_ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve sieve_extensions = +editheader +vacation-seconds +mboxmetadata +servermetadata +vnd.dovecot.debug sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_pipe_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } postmaster_address = postmaster(a)lerctr.org protocols = imap pop3 lmtp sieve recipient_delimiter = +-_ service anvil { unix_listener anvil { group = mail mode = 0666 } } service auth { unix_listener auth-client { mode = 0666 } unix_listener auth-master { mode = 0666 } } service doveadm { inet_listener http { port = 8080 ssl = yes } } service indexer-worker { drop_priv_before_exec = yes } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service stats { unix_listener stats-reader { group = mail mode = 0666 user = } unix_listener stats-writer { group = mail mode = 0666 user = } } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert = </home/ler/letsencrypt-home/*.lerctr.org/fullchain.cer ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+AESGCM:EECDH:EDH+AESGCM:EDH+aRSA:HIGH:!MEDIUM:!LOW:!aNULL:!eNULL:!LOW:!RC4:!MD5:!EXP:!PSK:!SRP:!DSS ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { args = username_format=%Ln /etc/passwd driver = passwd-file override_fields = domain=lerctr.org } verbose_proctitle = yes protocol lmtp { mail_plugins = " fts fts_solr notify virtual sieve mail_log" } protocol lda { mail_plugins = " fts fts_solr notify virtual sieve mail_log" } protocol pop3 { mail_plugins = " fts fts_solr notify virtual mail_log" } protocol !doveadm { mail_plugins = " fts fts_solr notify virtual mail_log" } protocol imap { imap_capability = +SPECIAL-USE imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags imap_logout_format = in=%i out=%o fhc=%{fetch_hdr_count} fhb=%{fetch_hdr_bytes} fbc=%{fetch_body_count} fbb=%{fetch_body_bytes} del=%{deleted} exp=%{expunged} trash=%{trashed} imap_metadata = yes mail_max_userip_connections = 50 mail_plugins = " fts fts_solr notify virtual mail_log imap_sieve imap_zlib" } -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

2 1

Case sensitive problem for vacation with dovecot lmtp pigeonhole
by kadafax＠gmail.com 11 Sep '19

11 Sep '19

Hi list, This issue was raised here by John Fawcett on 12/02/19 but got no response. So, as I'm facing the same issue, I would like to try again: When using the vacation sieve extension, if I send an email to user.name(a)mydomain.com it will work, but not to User.Name(a)mydomain.com. Is there a way to make the user part case insensitive so that the vacation auto-reply is fired whatever the way user part is written ? doveconf -n output: # 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.5 (2483b085) # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) ext4 # Hostname: myserver.mydomain.com auth_master_user_separator = * auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log default_client_limit = 10640 default_process_limit = 2128 default_vsz_limit = 0 hostname = mydomain.com info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = quota fts fts_lucene mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = maildir:/var/vmail/%Ln mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /usr/local/etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = /usr/local/etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. quota = maildir:User quota quota_rule = *:storage=30GB quota_rule2 = Trash:storage=+200M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/sieve-scripts/%Ln.sieve sieve_dir = /var/vmail/%Ln/home/sieve } postmaster_address = postmaster(a)mydomain.com protocols = imap pop3 lmtp sieve sieve quota_full_tempfail = yes service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0777 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 } service imap { process_limit = 2128 } service lmtp { executable = lmtp -L process_min_avail = 15 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = nobody } ssl_cert = </etc/letsencrypt/live/myserver.mydomain.com/fullchain.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota fts fts_lucene imap_quota } protocol lmtp { info_log_path = /var/log/dovecot-lmtp.log log_path = /var/log/dovecot-lmtp-errors.log mail_plugins = quota fts fts_lucene sieve } protocol sieve { mail_max_userip_connections = 20 }

2 2

Quota and maildir does not work with subfolders of INBOX
by Niels Kobschätzki 11 Sep '19

11 Sep '19

Hi, we just upgraded on FreeBSD (11.2-p14) from Dovecot 2.2 to Dovecot 2.3.7.2 and now we have kind of a big problem: the quota-plugin does not work correctly anymore. If a user has a subfolder of the INBOX (like .INBOX.foobar) the quota-calculation does not work anymore. It seems to be arbitrary but users get counted like double and more than they actually have in mails (the number of mails is correct, but the size gets calculated wrongly). The moment I remove those folders, the size gets calculated correctly. Unfortunately those folders are generated by some clients automatically afaik (like .INBOX.Trash) What can we do about that? There were already messages to the list about this in July 2018 but I didn't find any solution https://dovecot.org/pipermail/dovecot/2018-July/112439.html https://dovecot.org/pipermail/dovecot/2018-July/112440.html Niels

3 4

Off-site cloud backup (eg Amazon S3, Wasabi)
by James Brown 11 Sep '19

11 Sep '19

I’d like to synchronise our mail store to an Amazon S3 bucket or similar, for protection from ransomware. Server is running on macOS with maildir. Should I use a Dovecot process, rsync, a Mac app like Arq or Jungle Disk, or something else? Anyone have experience or opinions on this? Thanks, James.

3 2

Random duplicated emails
by Francis 11 Sep '19

11 Sep '19

Hello, I migrated our mail infrastructure to Dovecot on Ubuntu 18.04 some months ago. It works fine, but recently some users told me that they sometime receive duplicated emails. Same email content, same headers including message-id. I'm using two dovecot servers on two sites. Both server are in cluster. We don't use shared folders. All users that reported this issue so far are using the same server instance. The problematic emails are coming from local users on that instance too. The examples they given to me was emails with many recipients (To/CC). A specific message can be received twice (or more) by recipient A but only once by recipient B. I didn't see anything in the logs about sieve rules that redirect emails to others recipients. Where should I look to diagnostic this issue? Thanks. Server config: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) doveconf: Warning: service auth { client_limit=1000 } is lower than required under max. load (2500) doveconf: Warning: service anvil { client_limit=1000 } is lower than required under max. load (1503) # OS: Linux 4.15.0-60-generic x86_64 Ubuntu 18.04.3 LTS auth_default_realm = **edited** auth_master_user_separator = * auth_mechanisms = plain login default_process_limit = 500 first_valid_uid = 200 last_valid_uid = 200 mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication zlib quota" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox "Éléments envoyés" { special_use = \Sent } mailbox "Éléments supprimés" { special_use = \Trash } mailbox Brouillon { special_use = \Drafts } mailbox "Courrier indésirable" { special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes } plugin { mail_replica = tcps:**edited**:53683 quota = count:User quota quota_grace = 10%% quota_rule = *:storage=10G quota_rule2 = INBOX/Trash:storage=+100M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 La boite courriel du destinataire est pleine - Mailbox is full quota_status_success = DUNNO quota_vsizes = yes quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u quota_warning5 = storage=75%% quota-warning 75 %u quota_warning6 = -storage=80%% quota-warning '-80' %u quota_warning7 = -storage=100%% quota-warning '-100' %u sieve = /var/vmail/domains/%Ld/%Ln/.dovecot.sieve sieve_before = /var/vmail/sieve/before.sieve sieve_default = /var/vmail/sieve/default.sieve sieve_extensions = +vnd.dovecot.debug sieve_global = /var/vmail/sieve/global zlib_save = lz4 } protocols = " imap lmtp sieve" service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imap { port = 0 } process_min_avail = 10 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = **edited** port = 53684 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } } ssl = required ssl_cert = </etc/ssl/**edited**/**edited**.crtbundle ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " notify replication zlib quota sieve" postmaster_address = postmaster@**edited** } protocol imap { mail_plugins = " notify replication zlib quota imap_zlib imap_quota" } -- Francis

4 4

Dovecot Director upgrade from 2.2 to 2.3
by Alessio Cecchi 10 Sep '19

10 Sep '19

Hi, I have a setup with 3 Dovecot Director v2.2.36 and director_consistent_hashing = yes ;-) Now I would like to upgrade to 2.3.7, first only Director and after also Backend. Can works fine a ring of director with mixed 2.2 and 2.3 version? Mi idea is to setup a new Director server with 2.3, stop one server with 2.2 and insert the new 2.3 in the current ring to check if works fine. If all works fine I will replace all Director 2.2 with 2.3 version. Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 6

dovecot duplicate emails
by Битус Павел 10 Sep '19

10 Sep '19

Hello. I have problems with duplicate emails when using replication and configured message filters on the client. Looks like when letter comes, it processed by replication and immediately after this processed by filter on client and moved to another folder. When it arrives to slave server into INBOX, it replicated back to master into INBOX and only after that - slave receives MOVE command. The only thing I see in the log is that the letter comes 2 times with same msgid: Sep 10 15:01:31 lmtp(USERNAME(a)DOMAIN.com.ua)<9306><UD3fJJuQd11aJAAA8J4ONg>: Info: sieve: msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>: stored mail into mailbox 'INBOX' Sep 10 15:01:31 indexer-worker(USERNAME(a)DOMAIN.com.ua)<12248><UD3fJJuQd11aJAAA8J4ONg:GEsdNJuQd13YLwAA8J4ONg>: Info: Indexed 1 messages in INBOX (UIDs 55963..55963) Sep 10 15:01:32 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: copy from INBOX: box=INBOX.Jira, uid=5034, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:32 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: expunge: box=INBOX, uid=55963, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:32 indexer-worker(USERNAME(a)DOMAIN.com.ua)<12248><zqysnTGSU8tYUeJ+:oC6vHZyQd13YLwAA8J4ONg>: Info: Indexed 1 messages in INBOX.Jira (UIDs 5034..5034) Sep 10 15:01:33 indexer-worker(USERNAME(a)DOMAIN.com.ua)<12248><pQBKNpuQd13/OgAA8J4ONg:WIrkC52Qd13YLwAA8J4ONg>: Info: Indexed 1 messages in INBOX (UIDs 55964..55964) Sep 10 15:01:33 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: copy from INBOX: box=INBOX.Jira, uid=5035, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:33 imap(USERNAME(a)DOMAIN.com.ua)<9674><zqysnTGSU8tYUeJ+>: Info: expunge: box=INBOX, uid=55964, msgid=<f3701949-360e-e15d-3d76-7465f23fb0ba(a)DOMAIN.com.ua>, size=23393 Sep 10 15:01:34 indexer-worker(USERNAME(a)DOMAIN.com.ua)<9417><zqysnTGSU8tYUeJ+:gKqSOJ2Qd13JJAAA8J4ONg>: Info: Indexed 1 messages in INBOX.Jira (UIDs 5035..5035) Can you help me with this problem? dovecot --version 2.3.7.2 (3c910f64b) dovecot -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 4.19.66-gentoo x86_64 Gentoo Base System release 2.6 # Hostname: leon.DOMAIN.com.ua auth_cache_negative_ttl = 30 secs auth_cache_size = 256 M auth_cache_verify_password_with_worker = yes auth_gssapi_hostname = $ALL auth_krb5_keytab = /etc/krb5.keytab auth_mechanisms = gssapi ntlm plain login cram-md5 digest-md5 auth_worker_max_count = 2048 debug_log_path = /var/log/dovecot/dovecot_debug.log default_client_limit = 8492 default_vsz_limit = 2 G dict { lastlogin = mysql:/etc/dovecot/dovecot-dict-sql-lastlogin.conf } doveadm_password = # hidden, use -P to show it first_valid_uid = 8 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags listen = * log_path = /var/log/dovecot/dovecot.log mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_fsync = never mail_gid = mail mail_location = maildir:%h mail_max_keyword_length = 4096 mail_plugins = " notify replication zlib fts fts_solr mail_log notify" mail_uid = mail maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_lazy_writes = no metric imap_commands { event_name = imap_command_finished } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-ldap-people.conf.ext driver = ldap } passdb { args = /etc/dovecot/dovecot-ldap-calendar.conf.ext driver = ldap } plugin { fts = solr fts_autoindex = yes fts_autoindex_max_recent_msgs = 100 fts_solr = url=http://SOLR_IP_ADDRESS:8983/solr/dovecot/ last_login_dict = proxy::lastlogin mail_log_cached_only = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcp:REPLICATE.DOMAIN.com.ua:4092 replication_dsync_parameters = -d -N -l 30 -U replication_full_sync_interval = 7 days sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/default.sieve sieve_before2 = /etc/dovecot/sieve/default_spam.sieve zlib_save = gz zlib_save_level = 9 } postmaster_address = root(a)DOMAIN.com.ua protocols = imap pop3 lmtp sieve replication_max_conns = 50 service aggregator { fifo_listener replication-notify-fifo { user = mail } unix_listener replication-notify { user = mail } } service auth { unix_listener auth-client { mode = 0660 user = mail } unix_listener auth-master { mode = 0600 user = mail } unix_listener auth-userdb { mode = 0600 user = mail } } service config { unix_listener config { mode = 0600 user = mail } } service dict { unix_listener dict { group = mail mode = 0600 user = mail } } service doveadm { inet_listener { port = 4092 } } service imap-login { process_min_avail = 8 service_count = 0 vsz_limit = 256 M } service imap { client_limit = 10 process_limit = 4096 process_min_avail = 50 service_count = 100 } service lmtp { process_limit = 70 process_min_avail = 20 } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = mail mode = 0660 } vsz_limit = 20 G } ssl_cert = </etc/ssl/my_certs/fullchain1.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { args = /etc/dovecot/dovecot-ldap-people.conf.ext driver = ldap } userdb { args = uid=mail gid=mail home=/home/DOMAIN.com.ua/%Ln username_format=%Ln(a)DOMAIN.com.ua mail=maildir:/home/DOMAIN.com.ua/%Ln:UTF-8 driver = static } protocol lmtp { auth_username_format = %Lu mail_fsync = optimized mail_plugins = " notify replication zlib fts fts_solr mail_log notify sieve" } protocol lda { mail_fsync = optimized } protocol imap { imap_idle_notify_interval = 5 mins imap_max_line_length = 4 M imap_metadata = yes mail_max_userip_connections = 2048 mail_plugins = " notify replication zlib fts fts_solr mail_log notify fts fts_solr last_login" } protocol sieve { managesieve_max_line_length = 65536 }

1 0

New to dovecot admin, question about testing a dovecot configuration
by Gerben Wierda 10 Sep '19

10 Sep '19

I am new to dovecot administration, not new to postfix administration. When setting up a new postfix/dovecot combo from scratch, can I start with dovecot alone and still be able to test it? E.g. can I address the local delivery agent with some sort of test email? If so, how? I haven’t been able to find it in the wiki. Gerben Wierda Chess and the Art of Enterprise Architecture <https://ea.rna.nl/the-book/> Mastering ArchiMate <https://ea.rna.nl/the-book-edition-iii/> Architecture for Real Enterprises <https://www.infoworld.com/blog/architecture-for-real-enterprises/> at InfoWorld On Slippery Ice <https://eapj.org/on-slippery-ice/> at EAPJ

1 0

Force dovecot-uidlist reset
by Tom Sommer 09 Sep '19

09 Sep '19

Is there a way to force Dovecot to rebuild dovecot-uidlist from zero? It seems deleting all indexes and dovecot-* files followed by "doveadm force-resync" is not enough? It just gets the same UIDs? Perhaps from Maildir filenames? But I would like to reset the uid of all mails. Thanks -- Tom

1 1

doveadm mailbox list
by Daniel Miller 08 Sep '19

08 Sep '19

It's quite likely I'm doing it wrong, but... Given a valid mailbox... doveadm mailbox list -u <username> realmb returns "realmb" doveadm mailbox list -u <username> real* returns "realmb" Seems reasonable. Now, with a non-existent mailbox... doveadm mailbox list -u <username> bogus returns "bogus" doveadm mailbox list -u <username> bogus* returns "" Is this a bug or correct behavior? -- Daniel

2 1

TLS not working with iOS beta?
by Henrik Johansson 08 Sep '19

08 Sep '19

Hi, Have anyone else experienced problems using Dovecot with the mail app in beta releases of iOS/iPadOS 13? TLS is failing for my, it have worked fine for years and I am on the latest Dovecot version now, it works fine with older clients but not with the ones upgraded: Sep 04 19:49:16 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A Sep 04 19:49:16 imap-login: Debug: SSL alert: where=0x4008, ret=552: fatal handshake failure Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 04 19:49:16 imap-login: Debug: SSL error: SSL_accept() failed: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher Sep 04 19:49:16 imap-login: Debug: SSL: where=0x2002, ret=-1: error Sep 04 19:49:16 imap-login: Debug: SSL error: SSL_accept() failed: error:140800FF:SSL routines:ssl3_accept:unknown state Sep 04 19:49:16 imap-login: Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=11.22.33.44, lip=11.22.33.44, TLS handshaking: SSL_accept() failed: error:140800FF:SSL routines:ssl3_accept:unknown state, session=<lz7m2r2rjPouOvrk> Working client: Sep 04 19:57:58 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client hello A Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server hello A Sep 04 19:57:58 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write certificate A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write server done A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client certificate A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:01 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read client key exchange A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read certificate verify A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv3 read finished A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 read finished A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 write finished A Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2001, ret=1: SSLv3 flush data Sep 04 19:58:03 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully Sep 04 19:58:03 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully Sep 04 19:58:03 imap-login: Info: Login: user=<username>, method=LOGIN, rip=11.22.33.44, lip=11.22.33.44, mpid=28781, TLS, TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits), session=<LklR9r2Vtb4uOzra> Config: # egrep -v "^#|^$" 10-ssl.conf 10-auth.conf 10-ssl.conf:ssl = required 10-ssl.conf:ssl_cert = </path/to/fullchain.pem 10-ssl.conf:ssl_key = </path/to/privkey.pem 10-ssl.conf:ssl_dh = </path/to/dh.pem 10-ssl.conf:ssl_min_protocol = TLSv1.1 10-ssl.conf:ssl_cipher_list = ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH 10-auth.conf:disable_plaintext_auth = yes 10-auth.conf:auth_mechanisms = login 10-auth.conf:!include auth-system.conf.ext # dovecot --version 2.3.7.2 (3c910f64b) Anyone seeing something similar and/or have suggestions on how to proceed? Regards Henrik

4 7

intermittent failures when reuse_port is enabled
by Imre Rad 08 Sep '19

08 Sep '19

I'm encountering connection issues to Dovecot when reuse_port is activated. Some of the connections work fine, but about every 3rd of them does not receive the banner after the TCP connection has established: # nc -v 10.6.7.133 143 Connection to 10.6.7.133 143 port [tcp/imap2] succeeded! * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Hello. ^C # nc -v 10.6.7.133 143 Connection to 10.6.7.133 143 port [tcp/imap2] succeeded! * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Hello. ^C # nc -v 10.6.7.133 143 Connection to 10.6.7.133 143 port [tcp/imap2] succeeded! ^C The relevant processes listening on this port: # netstat -nap|grep 143 tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 15110/dovecot/imap- tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 15103/dovecot/imap- tcp 5 0 0.0.0.0:143 0.0.0.0:* LISTEN 15100/dovecot After removing the reuse_port = yes configuration directive there is only one single process listening (and the service is running without any issues): # netstat -nap|grep 143 tcp 0 0 0.0.0.0:143 0.0.0.0:* LISTEN 15292/dovecot The test config was Linux 4.9.0-4-amd64 on debian-stretch and dovecot-2.2.36.4 (compiled by me) running on a VM with 1 CPU core. If that matters, dovecot was started in a docker container but using the hosts's network namespace. What am I missing? Thanks, Imre

1 0

configure: error: fd passing is required for Dovecot to work
by zahn 06 Sep '19

06 Sep '19

Hello I changed our old gcc compiler from the version: gcc-4.1.2-13.fc6 to the newer one: gcc version 8.3.0 (GCC). Then I wanted to compile: Dovecot 2.3.7 and I get the following error with ./configure CPPFLAGS="-I/usr/local/openssl/include" \ LDFLAGS="-L/usr/local/openssl/lib" \ ./configure --prefix=/usr/local/dovecot \ --disable-static \ --with-ssl=openssl checking whether fd passing works... no configure: error: fd passing is required for Dovecot to work With the old Compiler configure works well. I couldn't find a solution on the Internet - can you help me ? -- Schöne Grüsse aus Oberdiessbach Martin Zahn Akadia AG Martin Zahn Software Ing. HTL Oracle Certificate Professional Information Technology Fichtenweg 10 CH-3672 Oberdiessbach Tel: +41 79 410 27 87 Web: www.akadia.com E-Mail: martin.zahn(a)akadia.com

3 2

Namespace structure
by Daniel Miller 06 Sep '19

06 Sep '19

Is the following "legal" for Dovecot? And...is this separation recommended or a bad idea? Particularly I'm asking about the "archives" namespace - I haven't actually implemented this yet and I'm checking before I break something. 10-mail.conf # Primary private namespace # Using sdbox for storage namespace inbox { type = private separator = / prefix = location = sdbox:/var/mail/%d/%n/sdbox inbox = yes hidden = no list = yes subscriptions = yes } # For long-term archival namespace archives { type = private separator = / prefix = Archives/ location = mdbox:/var/mail/%d/%n/Archives/mdbox subscriptions = no list = children } # Shared mailboxes mail_shared_explicit_inbox = yes namespace usershares { type = shared separator = / prefix = shared/%%n/ location = sdbox:/var/mail/%%d/%%n/sdbox subscriptions = no list = children } # Virtual mailboxes - for server-side searches namespace virtual { prefix = virtual/ separator = / location = virtual:/var/mail/%d/%n/virtual subscriptions = no list = children } -- Daniel

1 0

login failure after upgrade to dovecot 2.3.7.2
by MK 05 Sep '19

05 Sep '19

Hi List, after upgrading dovecot from version 2.2.33.2 to 2.3.7.2 I got a lot of error messages like the following if the process count of imap processes reaches nearly 1000 (~950 (there are a few pop-connections)). Sep 5 11:27:32 mailbox1 dovecot: imap-login: Internal login failure (pid=27728 id=116): user=<xxxx(a)xxxx.xx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, session=<qETa7cqRzpZV3IEw> Sep 5 11:27:33 mailbox1 dovecot: imap-login: Error: master(imap): net_connect_unix(imap) failed: Resource temporarily unavailable - http://wiki2.dovecot.org/SocketUnavailable (client-pid=27704, client-id=79, rip=x.x.x.x, created 511 msecs ago, received 0/4 bytes) Our setup consists of some servers with dovecot as proxy/redirector as frontend servers, redirecting the logged in users to the backend servers storing the mailboxes. OS is CentOS 6 on both server types. The backend server in question has usually ~ 2500 simultaneous imap connections. Before the update there where no problems. During the update I merged the old 2.2 config into the new 2.3 from the fresh installed binary (Repo from dovecot.org) As I found the failure above in the logfile I testet a lot, to figure out how to fix this, because for me it seemed to be some limit around 1000 connections/processes anywhere in the config. In the end I changed the default value of "default_client_limit" from ist default 1000 to 8192 and the problem was fixed for the moment. Now I'm trying to understand why this fixes the problem and what the difference is between 2.2 and 2.3. For me it seems like something changed in the architecture during the release change, but I didn't find anything. I looked into the release notes and also checked the differences beween the 2.2 and 2.3 default config values (using doveconf -d). Can anyone help? Here is the output of doveconf -n # 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.10 (Final) auth_mechanisms = plain login default_client_limit = 8192 disable_plaintext_auth = no mail_location = mdbox:%h mail_plugins = quota zlib managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::file:%h/dovecot-quota quota_warning = storage=90%% quota-warning 90 %u sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz zlib_save_level = 6 } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = imap pop3 lmtp sieve service imap-login { process_min_avail = 12 service_count = 0 } service imap { process_limit = 8192 } service lmtp { inet_listener lmtp { address = * port = 24 } } service pop3-login { process_min_avail = 12 service_count = 0 } service pop3 { process_limit = 1024 } service quota-warning { executable = script /scripts/quota.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = quota zlib sieve } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota zlib imap_quota } protocol pop3 { mail_plugins = quota zlib } greetings, Oliver

1 0

dsync and altpath on shared storage.
by Peter Mogensen 05 Sep '19

05 Sep '19

Hi, I was wondering... If one had mdbox ALT path set to a shared storage mount (say, on NFS) and one wanted to move a mailbox to a different host... I guess it in principle wouldn't be necessary to copy all the ALT storage through dsync, when the volume could just be mounted on the new host. Is there anyway for dsync to avoid moving Gigabytes of data for could just be "moved" by moving the mount? /Peter

2 6

Different passdb backends for different services
by lists＠mlserv.org 05 Sep '19

05 Sep '19

Hi, as Dovecot supports submission, which is the sending direction, I am interested to know, if I can configure a separate passdb backend just for submission. I habe LDAP attributes that differ sending or receiving permissions. It would be nice, if I had a second passdb backend just for submission, which has a LDAP filter for locking this service. Example for IMAP, POP3, Sieve: (&(mail=%s)(mailAllowIncoming=TRUE)) Example for Submission: (&(mail=%s)(mailAllowOutgoing=TRUE)) Use case is to lock compromised accounts that send spam. An operator can notify the account owner by placing a warning messegae into his/her mailbox. Is this possible? Thanks in advance Christian

3 9

Global ACL in SQL or LDAP possible?
by lists＠mlserv.org 04 Sep '19

04 Sep '19

Hi, Can global ACLs for public folders be stored in SQL or LDAP instead of a dovecot-acl file? I could only find something for shared folders, but not for public folders? Thanks in advance Christian

1 1

Server administration
by Aleksandr Mette 04 Sep '19

04 Sep '19

Dear Sirs, There is Postfix+Dovecot+Sogo installation in our company. I have attentively red Installation and Configuration Guide. However, I could not find some information. Could you give me an advise how to: 1. Add/remove e-mail address 2. Change user e-mail address password 3. Add user e-mail address into mail alias 4. Forward e-mail 5. List all users e-mails What is the best GUI interface for Dovecot administration? With kind regards, Alex

9 14

How are imap images supposed to be stored
by d.gentner＠m4ever.de 04 Sep '19

04 Sep '19

Hello. This is a question about the imap protocol, since I am having a hard time finding these things out, so I hope you guys can help me. We are using Dovecot to receive mails and having an issue with Thunderbird. Basically, we are running into this bug with a lot of people: https://bugzilla.mozilla.org/show_bug.cgi?id=216308. <https://bugzilla.mozilla.org/show_bug.cgi?id=216308> The gist of it is, that sometimes, when there is an image linked with the imap:// protocol in the mail, then Thunderbird wants to subscribe to the folder that contains this mail, but fails. If I pull the mail off the server, it also has that link in the raw mail. However, most mails store the images as a base64 code block. I have actually seen mails getting changed over night, from imap:// links to base64 code blocks, but I can't reproduce that. All computers are supposed to have the same Thunderbird version and they do. My questions right now are: * Speaking about the general imap/mail protocol, do mails store images as base64 or imap:// links. What is the standard here? * Is there anything we can do to prevent Thunderbird from running into this bug? We are using Dovecot 2.2.36.4 * What can cause an email getting changed over night or getting synced back with the base64 blocks? It was already read, when we ran into the issue. With best regards, Domeniko Gentner

2 1

Mailcrypt plugin private password
by info＠unkn0wn3d.com 04 Sep '19

04 Sep '19

Hello there, is there a way to make the mailcrypt plugin use the user's password or at least store it in a hashed value? I'm using a passwd file for authentication. I feel uncomfortable saving the private password in plaintext in that file. Regards

2 5

plugin for IMAP protocol to modifie any email
by Matthew Brown 04 Sep '19

04 Sep '19

is there a plugin or anyway I can modifie any email content ? I would like to change body text, headers, attachments

3 3

CVE-2019-11500: Critical vulnerability in Dovecot and Pigeonhole
by Aki Tuomi 04 Sep '19

04 Sep '19

Dear subscribers, we have been made aware of critical vulnerability in Dovecot and Pigeonhole. --- Open-Xchange Security Advisory 2019-08-14 Product: Dovecot Vendor: OX Software GmbH Internal reference: DOV-3278 Vulnerability type: Improper input validation (CWE-20) Vulnerable version: All versions prior to 2.3.7.2 and 2.2.36.4 Vulnerable component: IMAP and ManageSieve protocol parsers (before and after login) Report confidence: Confirmed Solution status: Fixed by Vendor Fixed version: 2.3.7.2, 2.2.36.4 Researcher credits: Nick Roessler and Rafi Rubin, University of Pennsylvania Vendor notification: 2019-04-13 Solution date: 2019-06-05 Public disclosure: 2019-08-28 CVE reference: CVE-2019-11500 CVSS: 8.1 (CVSS3.0/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:H/A:H) Vulnerability Details: IMAP and ManageSieve protocol parsers do not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Risk: This vulnerability allows for out-of-bounds writes to objects stored on the heap up to 8096 bytes in pre-login phase, and 65536 bytes post-login phase, allowing sufficiently skilled attacker to perform complicated attacks that can lead to leaking private information or remote code execution. Abuse of this bug is very difficult to observe, as it does not necessarily cause a crash. Attempts to abuse this bug are not directly evident from logs. Steps to reproduce: This bug is best observed using valgrind to see the out of bounds read with following snippet: perl -e 'print "a id (\"foo\" \"".("x"x1021)."\\A\" \"bar\" \"\000".("x"x1020)."\\A\")\n"' | nc localhost 143 Solution: Operators should update to the latest Patch Release. There is no workaround for the issue. --- Aki Tuomi Open-Xchange oy

8 14

Dovecot and Apple's Mail.app not playing nicely?
by Coy Hile 03 Sep '19

03 Sep '19

Hi all, Is there anything cute one has to take into account when using Dovecot with users of Apple’s Mail.app? Behavior I’m seeing is that if I delete or move messages via Webmail (Roundcube, Horde, or even ActiveSync via Mail.app on my phone), they do get moved or deleted. However, if I take the same actions in the desktop mail client, when logging in to the Webmail (or phone) app, I see the messages still seeming to be in the Inbox. Is this known behavior? A peculiarity in Apple Mail? -- Coy Hile coy.hile(a)coyhile.com

4 5

Re: Re: Quota count and clone questions
by Bratislav ILIC 03 Sep '19

03 Sep '19

Hi, I presume you figured this out after all this time, but in any case. Since quota_clone is not cumulative, it always inserts new record with actual quota, to be able to get correct results in postfixadmin one must: DROP TRIGGER mergequota2 ON quota2; And to prevent ERROR: duplicate key value violates unique constraint "quota2_pkey": CREATE FUNCTION public.clone_quota2() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN UPDATE quota2 SET bytes = NEW.bytes, messages = NEW.messages WHERE username = NEW.username; IF found THEN RETURN NULL; ELSE RETURN NEW; END IF; END; $$; And then: CREATE TRIGGER clonequota2 BEFORE INSERT ON quota2 FOR EACH ROW EXECUTE PROCEDURE public.clone_quota2() ; And finally just put: quota_clone_dict = proxy::sqlquota Hope this helps somebody trying to figure out this too. > I figured out that I can't just drop maintaining quota2 if I want > postfixadmin to report the quota status. > > I also figured out a way to run a test on my config guesses. I will try > to fit it in today, or tomorrow. But any advise to the questions below > are welcomed! > >>On 2/13/19 8:53 PM, Robert Moskowitz via dovecot wrote: >> all this almost reads like I can drop maintaining the quota2 table? >> >> From https://wiki.dovecot.org/Quota/Count >> >> mailbox_list_index = yes >> # Avoid spending excessive time waiting for the quota calculation to >> finish when >> # mails' vsizes aren't already cached. If this many mails are opened, >> finish the >> # quota calculation on background in indexer-worker process. Mail >> deliveries will >> # be assumed to succeed, and explicit quota lookups will return >> internal error. >> mail_vsize_bg_after_count = 100 >> >> seems to belong in 10-mail.conf. That is where those var are shown. >> >> But: >> >> >> plugin { >> # 10MB quota limit >> quota = count:User quota >> quota_rule = *:storage=10M >> >> # This is required - it uses "virtual sizes" rather than "physical >> sizes" for quota counting: >> quota_vsizes = yes >> } >> >> I am having problems with. Right now for quota I have: >> >> plugin { >> quota = dict:user::proxy::sqlquota >> trash = /etc/dovecot/dovecot-trash.conf.ext >> } >> >> How do I reconcile these two? >> >> Then for clone: https://wiki.dovecot.org/Plugins/QuotaClone >> >> how does: >> >> mail_plugins = $mail_plugins quota quota_clone >> plugin { >> quota_clone_dict = redis:host=127.0.0.1:port=6379 >> } >> >> get replaced with something for mysql? >> >> dovecot-sql.conf.ext: >> >> driver = mysql >> connect = host=/var/lib/mysql/mysql.sock dbname=postfix user=postfix >> password=$Postfix_Database_Password >> default_pass_scheme = $cryptsha-CRYPT >> # following should all be on one line. >> password_query = SELECT username as user, password, >> concat('/home/vmail/', maildir) as userdb_home, >> concat('maildir:/home/vmail/', maildir) as userdb_mail, 101 as >> userdb_uid, 12 as userdb_gid FROM mailbox WHERE username = '%u' AND >> active = '1' >> # following should all be on one line >> user_query = SELECT concat('/home/vmail/', maildir) as home, >> concat('maildir:/home/vmail/', maildir) as mail, 101 AS uid, 12 AS >> gid, CONCAT('*:messages=30000:bytes=', quota) as quota_rule FROM >> mailbox WHERE username = '%u' AND active = '1' >> >> and >> >> dovecot-dict-sql.conf.ext: >> >> connect = host=/var/lib/mysql/mysql.sock dbname=postfix user=postfix >> password=$Postfix_Database_Password >> map { >> pattern = priv/quota/storage >> table = quota2 >> username_field = username >> value_field = bytes >> } >> map { >> pattern = priv/quota/messages >> table = quota2 >> username_field = username >> value_field = messages >> } >> >> >> >> thanks

1 0

Dovecot 2.3.7 - char "-" missing
by Domenico Pastore 03 Sep '19

03 Sep '19

Hello, i have update dovecot from version 2.2.15 to 2.3.7.2. I have a problem with mine java software because there is a different response when open connection to doveadm. I need open socket to doveadm for get imap quota of a mailbox. With version 2.2.15: # telnet 192.160.10.4 924 Trying 192.160.10.4... Connected to 192.160.10.4. Escape character is '^]'. - With version 2.3.7.2: # telnet 192.160.10.3 924 Trying 192.160.10.3... Connected to 192.160.10.3. Escape character is '^]'. The difference is "-" character. The version 2.3.7 not respond with "-" character after opening the connection. Is it possible to add the character again with a parameter? Why did doveadm's answer change? Br Domenico

3 2

Segfault with error 4 in doveadm-server
by holger＠dehnhardt.org 03 Sep '19

03 Sep '19

I set up dovecot on two servers with replication enabled. The replication is over a vpn connection. I get massive segfaults from doveadm-server every few minutes. dmesg: doveadm-server[30030]: segfault at 0 ip 0000557fa16d2e62 sp 00007ffdcaafec50 error 4 in doveadm-server[557fa16a3000+41000] [486397.225636] Code: 40 03 4c 8d 3c c3 48 8d 44 24 18 48 89 44 24 08 0f 1f 84 00 00 00 00 00 49 8b bc 24 60 01 00 00 e8 e3 2a 00 00 48 89 44 24 18 <48> 8b 13 48 8d 35 c4 8d 00 00 48 89 c7 e8 2c 2b 00 00 31 c0 c6 45 mail.err dovecot: doveadm: Fatal: master: service(doveadm): child 29507 killed with signal 11 (core dumps disabled) I am absolutely unaware of how I can come to the cause of this error. Can anyone give me a hint? ----------------------------- Configuration: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 5.1.14-xen x86_64 Debian 8.11 auth_debug = yes disable_plaintext_auth = no doveadm_password = xxxx doveadm_port = 4711 listen = * mail_access_groups = vmail mail_gid = vmail mail_home = /var/vmail/%d/%n mail_location = maildir:~/mail mail_plugins = " notify replication" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcp:backup-vpn.local:12345 sieve = ~/.dovecot.sieve sieve_after = ~/sieve/after sieve_before = ~/sieve/before sieve_default = /var/vmail/%d/default.sieve sieve_dir = ~/sieve } protocols = " imap sieve pop3" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener auth-client { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { mode = 0666 user = vmail } } service config { unix_listener config { user = vmail } } service doveadm { client_limit = 1 idle_kill = 0 inet_listener { address = master-vpn.local port = 12345 } process_limit = 0 process_min_avail = 0 user = vmail } service replicator { client_limit = 0 drop_priv_before_exec = no idle_kill = 4294967295 secs process_limit = 1 process_min_avail = 0 service_count = 0 unix_listener replicator-doveadm { group = vmail mode = 0666 user = dovecot } vsz_limit = 8 G } ssl_cert = </etc/ssl/certs/xxx.cert ssl_key = </etc/ssl/private/xxx.key userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_line_length = 65536 } protocol lda { mail_plugins = " notify replication acl sieve" } protocol imap { mail_plugins = " notify replication acl" }

3 6

LMTP Post login script for acl_groups
by lists＠mlserv.org 03 Sep '19

03 Sep '19

Hi, I use a post login script for imap, to fetch acl groups from LDAP. Because Dovecot can only deal with a single value, which must be a comma seperated list of groups, I decided to use a post login script do deal with multi values in LDAP: This looks like this in LDAP: rnsMSACLGroup: admin rnsMSACLGroup: automx rnsMSACLGroup: amavis rnsMSACLGroup: postfix rnsMSACLGroup: dovecot rnsMSACLGroup: rspamd rnsMSACLGroup: powerdns rnsMSACLGroup: sogo rnsMSACLGroup: zabbix rnsMSACLGroup: dane-users rnsMSACLGroup: gentoo rnsMSACLGroup: openbsd My post login script looks like this: --------------------------------------------------------- #!/bin/sh BINDDN='cn=dovecot-postlogin,ou=people,ou=it,dc=roessner-net,dc=de' BINDPWFILE='/etc/dovecot/ldap-postlogin.secret' BASE='ou=people,ou=it,dc=roessner-net,dc=de' LDAPSEARCH="/usr/bin/ldapsearch" AWK="/usr/bin/awk" test -x ${LDAPSEARCH} || exec "$@" test -x ${AWK} || exec "$@" ACL_GROUPS=$( ${LDAPSEARCH} -LLL -ZZ -y ${BINDPWFILE} -xD ${BINDDN} -b ${BASE} "(rnsMSDovecotUser=${USER})" rnsMSACLGroup | \ grep rnsMSACLGroup | \ ${AWK} -vORS=, '{ print $2 }' | \ sed 's/,$/\n/' ) export ACL_GROUPS export USERDB_KEYS="${USERDB_KEYS} acl_groups" exec "$@" --------------------------------------------------------- This script is included in imap-postlogin executables and works for logged in users. But it does not work for LMTP. LMTP itself seems not to have any permissions to access the folders associated with these groups. I thought, I simply could add the imap-postlogin block to lmtp-postlogin and that would work, but it doesn't. So here is the question: What am I missing in Dovecot that LMTP can also have ACL_GROUPS like the imap service? Here is my config (non-defaults): --------------------------------------------------------- doveconf -n # 2.3.7.1 (0152c8b10): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.1 (db5c74be) # OS: Linux 4.19.44-gentoo x86_64 Gentoo Base System release 2.6 # Hostname: mx.roessner-net.de auth_cache_size = 64 M auth_master_user_separator = * auth_mechanisms = plain login auth_ssl_username_from_cert = yes auth_verbose = yes default_client_limit = 5000 default_process_limit = 500 default_vsz_limit = 512 M disable_plaintext_auth = no hostname = mail.roessner-net.de imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags imap_max_line_length = 4 M lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_rcpt_check_quota = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_access_groups = vmail mail_attachment_dir = /var/mail/virtual/copymail/attachments mail_gid = vmail mail_location = sdbox:~/sdbox mail_max_keyword_length = 4096 mail_plugins = quota acl fts fts_lucene zlib mail_log notify mail_privileged_group = mail mail_save_crlf = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds imapsieve vnd.dovecot.imapsieve mdbox_preallocate_space = yes mdbox_rotate_size = 128 M namespace { list = children location = sdbox:%%h/sdbox prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace { hidden = no list = children location = maildir:/var/mail/virtual/public:INDEXPVT=~/Maildir/public prefix = Public/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox "Deleted Messages" { special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk-E-Mail { special_use = \Junk } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/dovecot-acl:cache_secs=300 acl_shared_dict = file:/var/mail/virtual/shared-mailboxes.db fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. imapsieve_mailbox1_before = file:/etc/dovecot/sieve/rspamd.d/report-spam.sieve imapsieve_mailbox1_causes = COPY FLAG imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/rspamd.d/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge copy save mailbox_create mailbox_delete mailbox_rename mail_log_fields = box msgid quota = count:User quota quota_grace = 10%% quota_rule = *:storage=300M:messages=20000 quota_rule2 = Trash:storage=+500M quota_rule3 = Sent:storage=+2G quota_rule4 = Archive:storage=+2G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO quota_vsizes = yes quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u quota_warning3 = -storage=100%% quota-warning below %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/sieve/after sieve_before = /etc/dovecot/sieve/before sieve_extensions = +vacation-seconds sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute +vnd.dovecot.debug sieve_pipe_bin_dir = /usr/bin sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_default_period = 10d sieve_vacation_max_period = 30d sieve_vacation_min_period = 1h zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp submission sieve service auth-worker { extra_groups = ssl-cert unix_listener auth-worker { mode = 0600 user = vmail } user = vmail } service auth { extra_groups = ssl-cert unix_listener /var/spool/postfix-submission/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = vmail } service config { unix_listener config { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 127.0.0.1 134.255.226.248 ::1 2a05:bec0:28:1:134:255:226:248 } inet_listener imaps { port = 0 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-masteruser.sh /usr/local/bin/dovecot-lastlogin.sh /usr/local/bin/dovecot-aclgroups.sh user = vmail } service imap { executable = imap imap-postlogin } service lmtp-postlogin { executable = script-login /usr/local/bin/dovecot-aclgroups.sh user = vmail } service lmtp { executable = lmtp lmtp-postlogin inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener /var/spool/postfix/private/lmtp-dovecot { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { address = 127.0.0.1 134.255.226.248 ::1 2a05:bec0:28:1:134:255:226:248 } } service pop3-login { inet_listener pop3 { address = 127.0.0.1 134.255.226.248 ::1 2a05:bec0:28:1:134:255:226:248 } inet_listener pop3s { port = 0 } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = 127.0.0.1 port = 12340 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh extra_groups = mail unix_listener quota-warning { group = vmail mode = 0600 user = vmail } user = vmail } ssl_cert = </etc/ssl/mail.roessner-net.de/cert/fullchain.pem ssl_cipher_list = ALL:!DH:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH ssl_client_cert = </etc/ssl/mail.roessner-net.de/cert/fullchain.pem ssl_client_key = # hidden, use -P to show it ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes submission_client_workarounds = whitespace-before-path submission_relay_host = mail.roessner-net.de submission_relay_port = 5870 submission_relay_ssl = starttls submission_relay_trusted = yes userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = quota acl fts fts_lucene zlib mail_log notify sieve } protocol lda { mail_plugins = quota acl fts fts_lucene zlib mail_log notify sieve } protocol imap { mail_max_userip_connections = 50 mail_plugins = quota acl fts fts_lucene zlib mail_log notify imap_quota imap_acl imap_zlib imap_sieve } protocol submission { login_greeting = ESMTP } --------------------------------------------------------- Thanks for any help in advance Christian

3 15

dovecot 2.2 repo
by Bas van Rooijen 02 Sep '19

02 Sep '19

Hi, >From the v2.2.36.4 announcement message: "Binary packages are available at https://repo.dovecot.org/" My question is, where exactly? There's an instruction at repo.dovecot.org that tells you to replace 'ce-latest-2.3' with 'ce-<version>' but the given url doesn't even contain 'ce-latest-2.3' and http://repo.dovecot.org/ce-2.2.36 returns a 404. Thanks in advance! bvr.

4 3

Experimental docker images
by aki.tuomi＠dovecot.fi 02 Sep '19

02 Sep '19

Hi, you can now find experimental Docker images for dovecot CE releases (starting from 2.3.7.2) from https://hub.docker.com/r/dovecot/dovecot You can find instructions and more details on the repository page. ---- Aki

1 0

Re: Message delivered twice caused by an LMTP error "Got unexpected reply" during upgrade to 2.3
by Gabriele Nencioni 02 Sep '19

02 Sep '19

>>> On 08/09/2018 09:12 AM, Stephan Bosch wrote >>>>>> Can you make a pcap log of the LMTP communication between the two >>>>>> Dovecot hosts? That may give me a clue on which side of the >>>>>> communication is causing the issue. >>>>> Yes sure, where I can send it? >>>>> Here on list or at your address? >>>> >>>> Just send it to me. On 08/16/2018 10:21 AM, Stephan Bosch wrote: > Op 16/08/2018 om 09:46 schreef Gabriele Nencioni: >> On 08/15/2018 06:39 PM, Stephan Bosch wrote: >>> Op 15-8-2018 om 17:19 schreef Stephan Bosch: >>>> >>>> Op 13-8-2018 om 15:50 schreef Gabriele Nencioni: >>>>> So the scenario seems to be: >>>>> the error occurs between old directors and new backends >>>>> and between the new backends and the smtp submission_host (for >>>>> redirection) >>>>> and using the pipeline capability and attaching a file >>>>> >>>>> Let me know if you need more info >>>> In this case it is possible to obtain a debug log by enabling >>>> mail_debug=yes. Can you obtain a full debug log for such a failing >>>> submission session? >> Ok! It is attached. > > That is just weird. Dovecot reports that it cannot fully send the last > bits of the message and right thereafter the reply is received, which > Dovecot then interprets as out-of-sequence. The PCAP files tell me the > final CRLF.CRLF is actually sent to the other side, but somehow Dovecot > thinks it couldn't do that yet. > > Is this problem occurring for any and all SMTP client interactions, or > just every once in a while? Unfortunately it occurs just every once in a while and it is weird because at the second attempt the message is successful delivered or redirected. >>> Also, I don't see anything weird in the PCAP files: the command-reply >>> sequence is OK. This means that this is likely a bug in the current >>> Dovecot version. Those debug logs should tell me more. >> Ok! Which version of dovecot do you suggest to install? >> On debian current stable (stretch) repository there is the version >> number 2.2.27, have we to install it? > > If you're facing big trouble it is usually best to revert to the > previous version that worked I guess. Ok I'm agree! So, about you, which version can I try to install the 2.2.27 or the previous release of the 2.3 version such as v2.3.2 2018-06-29 or v2.3.1 2018-02-29? Regards, -- Gabriele Nencioni System Administrator eml gabriele.nencioni(a)register.it

2 6

purge causing reappearing mails?
by sch tel 02 Sep '19

02 Sep '19

3 3

flags not synced correctly with dovecot sync (dsync)
by Dan Christensen 31 Aug '19

31 Aug '19

I'm running dovecot 2.3.4.1 from https://repo.dovecot.org/ on Ubuntu 18.04 on three machines that I'll call server, laptop1 and laptop2. Both laptop1 and laptop2 run dovecot sync against server to keep local copies of my imap folders. Even when I initially had only two machines, laptop1 and server, I occasionally noticed that flags were lost, usually custom flags used by Gnus, but I couldn't reliably reproduce the problem. Now that I have two laptops syncing against the server, the problem has gotten worse and I figured out a way to reproduce it: - on server: create new IMAP folder test, and put two read messages in it - on laptop1: doveadm sync -u user -l 10 -m test -f user@server - on laptop2: doveadm sync -u user -l 10 -m test -f user@server At this point, all three machines show the two messages M1 and M2 as being read. - on laptop1: mark message M1 unread - on laptop2: mark message M2 unread - on laptop1: doveadm sync -u user -l 10 -m test -f user@server Both laptop1 and server have M1 unread, M2 read, as expected. - on laptop2: doveadm sync -u user -l 10 -m test -f user@server Now laptop2 and server have M1 *read*, M2 unread. - on laptop1: doveadm sync -u user -l 10 -m test -f user@server Now laptop1 and the server have both M1 and M2 *read*. - on laptop2: doveadm sync -u user -l 10 -m test -f user@server Now laptop2 has both read as well. The two lines that say "*read*" are wrong in my opinion. dsync propagated a read mark to an unread message, even though that message was marked unread more recently than it was marked read. I usually use stateful sync, and get many related problems. I just did a test in which M1 and M2 started out read, and I started with empty files named dstate.test on laptop1 and laptop2. Then I did the above procedure, using the command doveadm sync -u user -l 10 -m test -s "`cat dstate.test`" user@server > dstate.test At the end, laptop2 and server had both messages unread (which is good), but laptop1 had only M1 unread, and repeated runs of the sync command did not correct this. So the stateful sync failed to detect a change. Are these bugs in dovecot? Is there more information that I can provide? The output of doveconf -n on one machine is below, and the others are almost identical. Thanks for any help! Dan # 2.3.4.1 (3c0b8769e): /etc/dovecot/dovecot.conf # OS: Linux 4.15.0-45-generic x86_64 Ubuntu 18.04.1 LTS # Hostname: laptop2 auth_mechanisms = plain login listen = 127.0.0.1 mail_index_log2_max_age = 10 days mail_index_log_rotate_min_age = 1 days mail_index_log_rotate_min_size = 300 k mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } protocols = imap service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 943 ssl = yes } } service imap { process_limit = 25 } ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { postmaster_address = [elided] } protocol imap { mail_max_userip_connections = 20 }

3 8

Sieve Header question.
by Larry Rosenman 31 Aug '19

31 Aug '19

I'm trying to make my github processing better, but I'm missing something. I have the following: if address :all :contains "from" ["github.com"] { addflag "github"; addflag "MyFlags" "github"; set "mailbox" "GitHub"; if address :matches :user "to" "*" { set "GHUser" "${1}"; addflag "${GHUser}"; addflag "MyFlags" "${GHuser}"; } if header :matches "List-ID" "(.*/.*) <(.*)>" { set "mailbox" "github-lists/${1}"; } fileinto :flags "${MyFlags}" :create "${mailbox}"; stop; } I'm trying to match the apache-incubator-superset part of: List-ID: apache/incubator-superset <incubator-superset.apache.github.com> What am I missing? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

1 1

problems witch quota working
by Bartłomiej Solarz-Niesłuchowski 30 Aug '19

30 Aug '19

Good evening! Dear list I have problem with quota-fs (rquota) support with dovecot. I use dovecot version: dovecot --version 2.3.7.1 (0152c8b10) build options: dovecot --build-options Build options: ioloop=epoll notify=inotify openssl io_block_size=8192 SQL driver plugins: mysql postgresql sqlite Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap(plugin) passwd prefetch passwd-file sql the privatedevices is in service set as no: systemctl show -p PrivateDevices dovecot PrivateDevices=no but quota (rquota on nfsv4) does not work: doveadm quota get -u solarz \doveadm(solarz): Error: Failed to get quota resource STORAGE: quota-fs: quotactl(Q_GETQUOTA, oceanic:/home) failed: No such file or directory Quota name Type Value Limit % User quota STORAGE error error error quota from os works: quota solarz Disk quotas for user solarz (uid 1761): Filesystem blocks quota limit grace files quota limit grace /dev/mapper/vg01-lvol00 1588264 40480000 40480000 12383 0 0 oceanic:/home 76318340 86284030 87416830 561911 1190000 1650000 configuration: # OS: Linux 5.2.9-200.fc30.x86_64 x86_64 Fedora release 30 (Thirty) # Hostname: dervish.wsisiz.edu.pl auth_cache_size = 8 k mail_access_groups = mail mail_fsync = always mail_location = maildir:~/Maildir:INBOX=/var/spool/mail/%u mail_max_userip_connections = 500 mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = acl quota zlib trash mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/deny-users deny = yes driver = passwd-file } passdb { args = cache_key=#hidden_use-P_to_show# max_requests=256 driver = pam } passdb { args = cache_key=#hidden_use-P_to_show# max_requests=256 driver = pam } plugin { quota = fs:User quota quota_grace = 10%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster(a)wit.edu.pl protocols = imap pop3 service anvil { client_limit = 1155 } service auth { client_limit = 5248 user = root } service imap-login { client_limit = 1024 inet_listener imap { port = 0 } inet_listener imaps { address = [::], * port = 993 } process_limit = 1024 service_count = 0 } service imap { process_limit = 2048 vsz_limit = 512 M } service pop3-login { client_limit = 1024 inet_listener pop3 { port = 0 } inet_listener pop3s { address = [::], * port = 995 } process_limit = 128 service_count = 0 } service pop3 { process_limit = 2048 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = dovenull } ssl = required ssl_cert = </etc/pki/tls/certs/ipop3d.pem ssl_cipher_list = PROFILE=SYSTEM ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { driver = passwd } protocol imap { imap_client_workarounds = " tb-extra-mailbox-sep" mail_max_userip_connections = 200 mail_plugins = acl quota zlib trash imap_quota ssl_cert = </etc/pki/tls/certs/imapd.pem ssl_key = # hidden, use -P to show it } protocol pop3 { mail_max_userip_connections = 200 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv ssl_cert = </etc/pki/tls/certs/ipop3d.pem ssl_key = # hidden, use -P to show it } Thank for any clues. Best Regards -- Bartłomiej Solarz-Niesłuchowski, Administrator WSISiZ e-mail: Bartlomiej.Solarz-Niesluchowski(a)wit.edu.pl tel. 223486547, fax 223486501 JID: solarz(a)jabber.wit.edu.pl 01-447 Warszawa, ul. Newelska 6, pokój 421, pon.-pt. 8-16 Motto - Jak sobie pościelisz tak sie wyśpisz

1 0

doveadm backup mdbox - initial copy slow
by Gerald Galster 30 Aug '19

30 Aug '19

Hello, when calling doveadm backup like in the following example, it seems to flush every email to disk maxing out harddrives (iops) and resulting in very poor performance: Calling doveadm -o plugin/quota= backup -u "statusmails(a)domain.com" "mdbox:/backup/vmail/domain.com/statusmails" finished in 212 secs [changes:1800 copy:1800 delete:0 expunge:0] The source mdbox holds 1800 small emails and uses about 20 MB disk space only. I've tried -f (full sync) and -l 30 (locking the mailbox) which did not get faster, nor did doveadm sync -1. When using doveadm import all mails are copied in less than 3 seconds: time doveadm -o mail=mdbox:/backup/vmail/domain.com/statusmails import mdbox:/var/vmail/domain.com/statusmails "" all real 0m2.605s Are there any other options I could try to speed up doveadm backup, avoiding the flush after each email? This is doveadm from dovecot 2.2 (>= 2.2.33). Does anyone get better results with 2.3 tree? Best regards Gerald

2 2

User / Pass SQL queries
by Michael Hallager 29 Aug '19

29 Aug '19

Hi all, Is there any reason Dovecot shows the 'user' variable (ie: user(a)domain.tld) being obtained in the password query and not the (more logical) user query? Thanks, Michael

2 1

permission issue after yum update
by Voytek Eymont 29 Aug '19

29 Aug '19

I've just updated my dovecot on centos, after 'yum update' dovecot failed to work, with manual restart it worked fine, just giving a warning: # dovecot Warning: Corrected permissions for login directory /var/run/dovecot/token-login dovecot --version 2.3.7.2 (3c910f64b) all's good, just reporting in case this is relevant to rpm maintainers thanks, V

1 0

download Dovecot
by Miquel Demur 29 Aug '19

29 Aug '19

I have downloaded in MacMini MacOS X applications directory the Dovecot.tzr.gz file from Dovecot v2.3.7.2 in https://www.dovecot.org/download I have clicked on dovecot-2.3.7.2 tar.gz and it creates a dovecot-2.3.7.2 directory containung 35 files. https://wiki2.dovecot.org/FrontPage?action=show&redirect=PageD%27Accueil#Do… https://wiki2.dovecot.org/FrontPage?action=show&redirect=PageD%27Accueil#Do… I have to compile to install in Terminal Application? ./configure make sudo make install in https://wiki2.dovecot.org/CompilingSource#Compiling_Dovecot_From_Sources without reference for dovecot.

2 1

Pigeonhole release v0.4.24.2
by Aki Tuomi 29 Aug '19

29 Aug '19

Hi! We are pleased to release Pigeonhole release v0.4.24.2 Tarball is available at https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.2… https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.2… Changes ------- * CVE-2019-11500: ManageSieve protocol parser does not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Found by Nick Roessler and Rafi Rubin. --- Aki Tuomi Open-Xchange oy

2 2

Feature wishlist: Allow to hide client IP/host in submission service
by Sebastian Krause 28 Aug '19

28 Aug '19

Hi, In many mail setups a required feature (for privacy reasons) is to hide the host and IP of clients (in the "Received" header) that use the authenticated submission over port 587. In Postfix that's possible (https://serverfault.com/q/413533/86332) but not very nice to configure especially if you only want want to strip the Received header for port 587 submissions, but not on port 25. As far as I can see this configuration is not possible at all in the Dovecot submission server because the function which adds the Received header with the client's IP address (smtp_server_transaction_write_trace_record) is always called in submission-commands.c. It would be very useful if the submission server could anonymize the client with a single configuration option, then all the Postfix configuration mess (and using SASL) could be skipped by simply using the Dovecot submission server instead. The anonymization would work by replacing the client's EHLO host with "submission" and the IP address with 127.0.0.1. In full the Received header would look something like this where the first line is always the same: Received: from submission (unknown [127.0.0.1]) by mail.example.com with ESMTPSA id 8bV9D+51Yl1FOwAA1ctoJQ (envelope-from <john(a)example.com>) for <jane(a)example.com>; Sun, 25 Aug 2019 13:50:06 +0200 Sebastian

4 6

Pigeonhole release v0.5.7.2
by Aki Tuomi 28 Aug '19

28 Aug '19

Hi! We are pleased to release Pigeonhole release v0.5.7.2 Tarball is available at https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.2.… https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.2.… Binary packages are available at https://repo.dovecot.org/ Changes ------- * CVE-2019-11500: ManageSieve protocol parser does not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Found by Nick Roessler and Rafi Rubin. --- Aki Tuomi Open-Xchange oy

1 0

Dovecot release v2.2.36.4
by Aki Tuomi 28 Aug '19

28 Aug '19

Hi! We are pleased to release Dovecot release v2.2.36.4 Tarball is available at https://dovecot.org/releases/2.2/dovecot-2.2.36.4.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.36.4.tar.gz.sig Binary packages are available at https://repo.dovecot.org/ Changes ------- * CVE-2019-11500: IMAP protocol parser does not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Found by Nick Roessler and Rafi Rubin. --- Aki Tuomi Open-Xchange oy

1 0

Dovecot release v2.3.7.2
by Aki Tuomi 28 Aug '19

28 Aug '19

Hi! We are pleased to release Dovecot release v2.3.7.2 Tarball is available at https://dovecot.org/releases/2.3/dovecot-2.3.7.2.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.7.2.tar.gz.sig Binary packages are available at https://repo.dovecot.org/ Changes ------- * CVE-2019-11500: IMAP protocol parser does not properly handle NUL byte when scanning data in quoted strings, leading to out of bounds heap memory writes. Found by Nick Roessler and Rafi Rubin. --- Aki Tuomi Open-Xchange oy

1 0

unsubscrib
by Carlos Xavier 27 Aug '19

27 Aug '19

1 0

Group emails based on custom header
by Giovanni Mancuso 27 Aug '19

27 Aug '19

Hi, Can i "group by" e-mails based on custom header in imap? According to my experience if i use "XX UID SEARCH INTHREAD REFS HEADER myheader myvalue" i receive emails UID gathers by HEADER+VALUE, but in this way i need to fetch all emails using several imap SEARCH. Has dovecot any features that would help me do this? Thank you!!

2 1

doveadm mailbox handling for directory only
by sch tel 27 Aug '19

27 Aug '19

1 1

non-terminating processes
by Randy Bush 26 Aug '19

26 Aug '19

FreeBSD psg.com 11.3-RELEASE-p3 FreeBSD 11.3-RELEASE-p3 dovecot 2.3.6 (7eab80676) (latest from binary package system) symptom: multiple imap4s users complain that they can not fetch mail stopping and restarting dovecot is a temporary workaround dovenull 3713 0.0 0.0 11036 6816 - I 02:27 0:00.17 dovecot/imap-login user19 3714 0.0 0.0 9464 4756 - S 02:27 0:00.46 dovecot/imap dovenull 6261 0.0 0.0 11036 6812 - I 02:47 0:00.12 dovecot/imap-login dovenull 6262 0.0 0.0 11036 6804 - I 02:47 0:00.09 dovecot/imap-login dovenull 6263 0.0 0.0 11036 6804 - I 02:47 0:00.08 dovecot/imap-login user00 6264 0.0 0.0 9476 4924 - S 02:47 0:00.27 dovecot/imap user00 6265 0.0 0.0 9464 5444 - S 02:47 0:00.91 dovecot/imap user1 6266 0.0 0.0 9484 4636 - S 02:47 0:00.16 dovecot/imap dovenull 6277 0.0 0.0 11036 6804 - I 02:47 0:00.08 dovecot/imap-login user1 6279 0.0 0.0 9940 5096 - S 02:47 0:00.69 dovecot/imap dovenull 54595 0.0 0.0 11036 6916 - I 09:24 0:00.04 dovecot/imap-login dovenull 54596 0.0 0.0 11036 6916 - I 09:24 0:00.04 dovecot/imap-login user19 54597 0.0 0.0 9464 4784 - S 09:24 0:00.06 dovecot/imap user19 54598 0.0 0.0 9468 4676 - S 09:24 0:00.05 dovecot/imap dovenull 67450 0.0 0.0 11036 6924 - I 11:13 0:00.05 dovecot/imap-login userid0 67453 0.0 0.0 9488 4844 - S 11:13 0:00.05 dovecot/imap dovecot 68853 0.0 0.0 11484 6920 - S 11:29 0:00.32 dovecot/auth dovenull 74048 0.0 0.0 11036 6916 - I 12:15 0:00.03 dovecot/imap-login user54 74050 0.0 0.0 9552 4944 - I 12:15 0:00.02 dovecot/imap dovenull 74054 0.0 0.0 11036 6916 - I 12:15 0:00.03 dovecot/imap-login user54 74056 0.0 0.0 9492 5152 - I 12:15 0:00.06 dovecot/imap dovenull 74582 0.0 0.0 11036 6932 - S 12:20 0:00.07 dovecot/imap-login user19 74586 0.0 0.0 13572 7632 - S 12:20 0:04.14 dovecot/imap dovenull 74670 0.0 0.0 11036 6932 - S 12:21 0:00.05 dovecot/imap-login user01 74674 0.0 0.0 16768 6500 - S 12:21 0:02.33 dovecot/imap dovenull 74770 0.0 0.0 11036 6932 - S 12:21 0:00.05 dovecot/imap-login user01 74772 0.0 0.0 11500 5756 - S 12:21 0:00.08 dovecot/imap dovenull 74782 0.0 0.0 11036 6932 - I 12:21 0:00.04 dovecot/imap-login user01 74786 0.0 0.0 9460 5516 - I 12:21 0:00.05 dovecot/imap dovenull 74893 0.0 0.0 11036 6936 - I 12:22 0:00.04 dovecot/imap-login id0 74894 0.0 0.0 10644 5552 - S 12:22 0:00.05 dovecot/imap dovenull 74952 0.0 0.0 11036 6928 - I 12:23 0:00.03 dovecot/imap-login id0en 74954 0.0 0.0 9480 4900 - S 12:23 0:00.05 dovecot/imap dovenull 74958 0.0 0.0 11036 6916 - I 12:23 0:00.03 dovecot/imap-login id002 74959 0.0 0.0 9452 4756 - S 12:23 0:00.04 dovecot/imap dovenull 74961 0.0 0.0 11036 6916 - I 12:23 0:00.03 dovecot/imap-login id0 74962 0.0 0.0 9572 5088 - S 12:23 0:00.05 dovecot/imap dovenull 74964 0.0 0.0 11036 6916 - I 12:23 0:00.03 dovecot/imap-login id0 74966 0.0 0.0 9528 4892 - S 12:23 0:00.02 dovecot/imap dovenull 76400 0.0 0.0 11036 6916 - I 12:35 0:00.03 dovecot/imap-login user19 76401 0.0 0.0 9468 4804 - I 12:35 0:00.01 dovecot/imap root 76686 0.0 0.0 11484 7072 - S 12:38 0:00.37 dovecot/auth -w dovenull 77507 0.0 0.0 11036 6940 - I 12:45 0:00.16 dovecot/imap-login user5 77511 0.0 0.1 69164 12120 - I 12:45 0:00.49 dovecot/imap dovenull 78745 0.0 0.0 11036 6932 - S 12:56 0:00.05 dovecot/imap-login dovenull 78748 0.0 0.0 11036 6932 - S 12:56 0:00.06 dovecot/imap-login user5 78761 0.0 0.0 9476 5268 - S 12:56 0:00.12 dovecot/imap user5 78766 0.0 0.0 9492 5212 - S 12:56 0:00.12 dovecot/imap dovenull 78776 0.0 0.0 11036 6916 - S 12:56 0:00.03 dovecot/imap-login usr4 78777 0.0 0.0 9452 4564 - I 12:56 0:00.01 dovecot/imap root 88537 0.0 0.0 7940 1648 - Ss 00:51 0:04.00 /usr/local/sbin/dovecot -c /usr/local/etc/dovecot/dovecot.conf dovecot 88538 0.0 0.0 7880 1832 - S 00:51 0:01.04 dovecot/anvil root 88539 0.0 0.0 7880 1820 - S 00:51 0:00.86 dovecot/log dovenull 88540 0.0 0.0 11036 3992 - S 00:51 0:00.10 dovecot/imap-login root 88541 0.0 0.0 8008 3176 - S 00:51 0:04.18 dovecot/config dovecot 88542 0.0 0.0 7880 2592 - S 00:51 0:01.57 dovecot/stats dovenull 88544 0.0 0.0 11036 4088 - S 00:51 0:00.16 dovecot/imap-login dovenull 88545 0.0 0.0 11036 4064 - S 00:51 0:00.17 dovecot/imap-login dovenull 88546 0.0 0.0 11036 4016 - I 00:51 0:00.11 dovecot/imap-login user00 88547 0.0 0.0 9476 4156 - S 00:51 0:00.45 dovecot/imap usr9 88548 0.0 0.0 9460 4168 - S 00:51 0:00.49 dovecot/imap user1 88551 0.0 0.0 9452 3148 - S 00:51 0:00.22 dovecot/imap dovenull 88557 0.0 0.0 11036 4024 - I 00:52 0:00.11 dovecot/imap-login user17 88561 0.0 0.0 9472 3128 - S 00:52 0:00.22 dovecot/imap user17 88565 0.0 0.0 9472 3120 - S 00:52 0:00.23 dovecot/imap dovenull 88576 0.0 0.0 11036 3776 - I 00:52 0:00.18 dovecot/imap-login user19 88579 0.0 0.0 9604 2872 - S 00:52 0:00.40 dovecot/imap clues? just installing 2.3.7 from source to see if that helps randy

2 2

sometimes no shared cipher after upgrade from 2.2 to 2.3
by Kristijan Savic - ratiokontakt GmbH 23 Aug '19

23 Aug '19

We recently upgraded from dovecot 2.2 to 2.3.7.1-1 Not many, but some users are experiencing difficulties. The dovecot directors log: Aug 21 14:28:49 director01 dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=redacted, lip=10.0.0.120, TLS handshaking: SSL_accept() failed: error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher, session=<EeoTt5+QXwVUl87W> Any ide what could be causing it? Thanks, Kristijan

4 6

Trying to install Mailcrypt, receive completely blank emails
by sewing 22 Aug '19

22 Aug '19

Hi - perhaps someone can help me. Starting from a good and well-functioning mailserver setup, I have installed MailCrypt as per the instructions (I think) to have the per-user passworded keys setup. dovecot.conf /Added:/ mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_plugins = $mail_plugins mail_crypt dovecot-sql.conf.ext /commented this/ #password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; /Added this/ password_query = SELECT \ email as user, password, \ '%w' AS userdb_mail_crypt_private_password \ FROM virtual_users WHERE email='%u'; 90-plugin.conf /Updated this section/ plugin { mail_crypt_curve = secp521r1 mail_crypt_save_version = 2 /Have also tried all configurations this option set /#mail_crypt_require_encrypted_user_key = yes } /The user is in the mailserver SQL database:/ | user | password | userdb_mail_crypt_private_password | +----------------------+------------------------------------------------------------------------------------------------------------+------------------------------------+ | user@domain | 4t63ttt36tttttt3rt6r763r76t8998t858t4y48ht4huu4hti.76876t847t47yt4ty478yt4wgwygfwffweggggggggggwy8yey83//FElgiApZU. | %w /I have reissued the keys as other problem resolutions have instructed to do, and I can see the new keys are active./ doveadm -o plugin/mail_crypt_private_password=(same password as user@domain when created in SQL) mailbox cryptokey generate -u user@domain -UR /I have restarted dovecot and postfix each time./ /I then send an email either from the user@domain > user@domain OR otheruser@otherdomain > user@domain from Outlook. Authentication is fine (POP and SMTP). No errors appear in any logs - the logs show the mail as being received and I can see the mail in the user@domain /cur folder. If I try to Postcat any of these messages I get:/ postcat '1*********2.M**********3.(host),S=633,W=649:2,S' *** ENVELOPE RECORDS 1*********2.M*********3.(host),S=633,W=649:2,S *** message_size: YPTED postcat: fatal: invalid size record: YPTED??? */Within a minute or so I receive the email in the Outlook Inbox - except it is completely empty of anything. There is no TO / FROM/ SUBJECT / Body or even Routing information. See screencaps from Outlook/* <http://dovecot.2317879.n4.nabble.com/file/t5785/message_list_view.jpg> <http://dovecot.2317879.n4.nabble.com/file/t5785/empty-emails.jpg> /This happens with existing user mailboxes and new test mailboxes I create. If I deactivate Mailcrypt all mailboxes/user-accounts work fine. Any ideas??? I would like to get the working - thanks in advance for your expertise and time!/ Graham -- Sent from: http://dovecot.2317879.n4.nabble.com/

1 0

imapsieve suddenly not working anymore
by Thomas Stein 21 Aug '19

21 Aug '19

Hello one and all. Dovecot version 2.3.7.1 I've configured imapsieve like https://wiki.dovecot.org/HowTo/AntispamWithSieve a while a go and it worked for years now. Suddenly i noticed moving mails to the spamfolder does not trigger the report-spam.sieve script anymore. sieve-test gives: ~/.maildir/.Spam/cur $ sieve-test /usr/share/dovecot/sieve/report-spam.sieve 1542388745.M99384P16720.meine-oma.de\,S\=8173\,W\=8373\:2\,S -D sieve-test(himbeere): Debug: sieve: Pigeonhole version 0.5.7.1 (db5c74be) initializing sieve-test(himbeere): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. sieve-test(himbeere): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.7.1 (db5c74be) loaded sieve-test(himbeere): Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.7.1 (db5c74be) loaded debug: file storage: Using Sieve script path: /usr/share/dovecot/sieve/report-spam.sieve. debug: file script: Opened script `report-spam' from `/usr/share/dovecot/sieve/report-spam.sieve'. debug: Script binary /usr/share/dovecot/sieve/report-spam.svbin successfully loaded. debug: binary save: not saving binary /usr/share/dovecot/sieve/report-spam.svbin, because it is already stored. report-spam: error: the imapsieve extension cannot be used outside IMAP. sieve-test(himbeere): Info: final result: failed; resolved with successful implicit keep ~/.maildir/.Spam/cur $ I'm not sure the "the imapsieve extension cannot be used outside IMAP" is the error already or thats only because the sieve-test script. Any ideas on that? cheers, t.

1 3

maildir very dirty sync option
by Yousif Alkhateeb 20 Aug '19

20 Aug '19

Hello , I have an active passive dovecot setup with glusterfs as a mail storage and using maildirs, we used to have a problem when users with large mailboxes sync their folders. This caused the load average to increase in the server , after a while we have enabled the very_dirty_sync option in dovecot , things got better and the problem disappeared but we need to know if there is any thing else that we need to do or know about the very_dirty_sync option that may cause future problems . Thanks in advance

3 2

Dovecot and Oauth
by Michael Hallager 20 Aug '19

20 Aug '19

Hi all, I am considering expanding a mail server to support Oauth with Google. I have read through the following: https://wiki.dovecot.org/PasswordDatabase/oauth2 however, it doesn't work and appears to me to be missing important information, the least of which is API authorisation. Searching with Google, I can't find any more information beyond what amounts to C&P'ing of the above link. Has anyone successfully deployed Oauth with Google and could they please send me their config files? Thanks, Michael

2 1

Dovecot and hard links?
by ＠lbutlr 18 Aug '19

18 Aug '19

I was looking at a mail folder and I noted that a file in the inbox had a total of 11 hard links to it: 👹 # exa -lH cur/1564249738.M167990P53982.mail.covisp.net,S=8572,W=8738:2,Sgl .rw------- 11 8.6k vpopmail 19 Jul 19:59 cur/1564249738.M167990P53982.mail.covisp.net,S=8572,W=8738:2,Sgl 👹 # find . -samefile cur/1564249738.M167990P53982.mail.covisp.net,S=8572,W=8738:2,Sgl [6:59] [/usr/local/virtual/kreme(a)kreme.com/Maildir] ./.Junk/cur/1564253637.M808919P50422.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1564253637.M808920P50422.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1564253723.M977674P50422.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1564253723.M977675P50422.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1564253723.M977676P50422.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1563587951.M591499P47868.mail.covisp.net,S=8572,W=8738:2, ./.Junk/cur/1564851795.M381461P10380.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1564851795.M381462P10380.mail.covisp.net,S=8572,W=8738:2,bc ./.Junk/cur/1564852369.M760020P10380.mail.covisp.net,S=8572,W=8738:2,bc ./.Trash/cur/1563587951.M591499P47868.mail.covisp.net,S=8572,W=8738:2,S ./cur/1564249738.M167990P53982.mail.covisp.net,S=8572,W=8738:2,Sgl A bit of background: a few weeks ago I had thousands of files in the mail store that had hundreds of links, and I got that all cleaned up through juggling some grep and awk to delete all but one of the linked files. I figured this was caused by something I’d done when I moved local users into being virtual users and eliminated pam authentication in favor of sql. But this gives me pause. I think it’s normal for a file in the INBOX that is marked as junk or as deleted to have a single hard link (or at least it is since the default was set to maildir_copy_with_hardlinks), but 11? And when do these hard links get cleaned off? If I delete the trashed and junked messages, when does dovecot go through and removed the linked files in the cur/ folder? (It’s not immediate, I trashed those copies in Junk and Trash and the original file is still in the cur folder) -- There is nothing so stupid that some person somewhere will not, with earnestness, say it.

3 5

Re: Submission and TLS client certificate
by Christian Rößner 18 Aug '19

18 Aug '19

Hi, > Am 18.08.2019 um 16:52 schrieb Stephan Bosch via dovecot <dovecot(a)dovecot.org>: > > > >>> On 18/08/2019 10:09, Christian Rößner via dovecot wrote: >> Hi, >> is there some configuration parameter in Dovecot, which sends a TLS client certificate to the SMTP server? I would need this to have XCLIENT enabled and TLS with Postfix. This way I could permit sending based on the certificates fingerprint. > > Can you elaborate? Are you talking about the submission relay service or submission of outgoing messages from e.g. Sieve? The submission relay service. I like the idea of this service. Currently I did not activate the relay trust option, because I have no idea on how I could give permissions on the Postfix side. If Dovecot would send the certificate as reply to the smtpd_ask_ccert option, I could turn on XCLIENT in Dovecot and give permission based on the certificate fingerprint. At the moment I do STARTTLS from Dovecot to Postfix and have added Dovecot‘s IP to mynetworks in Postfix. But I would prefer XCLIENT. Thanks in advance Christian > > Regards, > > Stephan. > >> Thanks in advance >> Christian >> Von unterwegs aus gesendet >> -- >> Rößner-Network-Solutions >> Karl-Bröger-Str. 10, 36304 Alsfeld <x-apple-data-detectors://0/0> >> Fax: +49 6631 78823409 <tel:+49%206631%2078823409>, Mobil: +49 171 9905345 <tel:+49%20171%209905345> >> USt-IdNr.: DE225643613, https://roessner.website <https://roessner.website/> >> PGP Fingerprint: 8FB3 132F 85D8 C9C7 A9F1 9A3F 5183 D46C B885 897E

1 0

Submission and TLS client certificate
by Christian Rößner 18 Aug '19

18 Aug '19

Hi, is there some configuration parameter in Dovecot, which sends a TLS client certificate to the SMTP server? I would need this to have XCLIENT enabled and TLS with Postfix. This way I could permit sending based on the certificates fingerprint. Thanks in advance Christian Von unterwegs aus gesendet -- Rößner-Network-Solutions Karl-Bröger-Str. 10, 36304 Alsfeld Fax: +49 6631 78823409, Mobil: +49 171 9905345 USt-IdNr.: DE225643613, https://roessner.website PGP Fingerprint: 8FB3 132F 85D8 C9C7 A9F1 9A3F 5183 D46C B885 897E

2 1

exim+dovecot+sieve issue
by JM 18 Aug '19

18 Aug '19

Hi, We have mail server in production, with Exim + Dovecot + Mailman + OpenLDAP + Roundcube webmail server. Openldap keeps mail accounts in db, exim uses lmtp transport to deliver messages to dovecot. Mailman is responsible for mailing lists, Roundcube webmail has managsieve module to create sieve filters. Exim router,transport for dovecot > localuser: > driver = accept > domains = +local_domains : +virt_domains > check_local_user > transport = dovecot_lmtp > cannot_route_message = Unknown account > no_more > dovecot_lmtp: > driver = lmtp > socket = /var/spool/dovecot/lmtp > batch_max = 500 > rcpt_include_affixes It worked perfectly, but after upgrade from Dovecot 2.2 to Dovecot 2.3 major version, I've found a lot of strange errors in exim panic.log > mail exim[7821] Failed to get user name for uid 1175 > mail exim[7880]: Failed to get user name for uid 1173 etc where uids are ldap accounts and recognized by system (id 1175 shows user's name and group) Even worse, some clients start to receive notify email with strange content, when sending email to mailing lists > user(a)domain.com > Delay reason: error in redirect data: malformed address: > someuser1(a)domain.com may not follow someuser2(a)domain.com in " > someuser3(a)domain.com ..some crap here.." , where someuser addresses are > not related to recipient address and unknown for sender. After some investigation,I've found out that dovecot's sieve produces these errors (I suspect sieve vacation, when user creates vacation via Roundcube webmail). If set "submission_host = exim ip" in dovecot.conf, exim doesn't produce errors in panic.log anymore, but clients still receive email with malformed address. Please help me to find the root of the problem (some broken sieve filter in user's maildir, misconfiguration or breaking changes in new Dovecot version?) Current software Gentoo with 4.14.90 kernel Dovecot-2.3.7.1 Exim-4.92.1 Mailman 2.1.29 Openldap-2.4.47 Roundcube-1.3.8 Dovecot.conf attached Regards, Juri

2 1

2.3.7 + stats
by Matt Bryant 17 Aug '19

17 Aug '19

Is there any additional documentation/information around the new stats module. Have added some metrics just to see what they produce ## ## Metrics ### metric imap { event_name = imap_command_finished #source_location = example.c:123 #categories = fields = name args running_usecs bytes_in bytes_out #filter { # field_key = wildcard #} } metric sql { event_name = sql_query_finished } metric auth { event_name = auth_request_finished fields = user transport error successful } and get the following [root@stargate dovecot]# doveadm stats dump metric_name field count sum min max avg median stddev %95 imap duration 370 2007904499 130 62955249 5426768.92 2068 16436817.37 60026465 imap name 370 0 0 0 0.00 0 0.00 0 imap args 0 0 0 0 0.00 0 0.00 0 imap running_usecs 370 2007865330 81 62955127 5426663.05 1991 16436816.76 60026329 imap bytes_in 370 5366 2 173 14.50 8 19.04 35 imap bytes_out 370 2119971 0 941517 5729.65 41 53760.89 2082 sql duration 18 28991 991 2305 1610.61 1660 377.36 2305 auth duration 12 26046980 8146 7079879 2170581.67 847730 2457811.23 7079879 auth user 12 0 0 0 0.00 0 0.00 0 auth transport 12 0 0 0 0.00 0 0.00 0 auth error 0 0 0 0 0.00 0 0.00 0 auth successful 0 0 0 0 0.00 0 0.00 0 the main wiki page on stats/events doesnt really hold much detail whats stores for each event the above fields dont make much sense and top no longer works out of the box [root@stargate dovecot]# doveadm stats top usage: doveadm [-Dv] [-f <formatter>] stats <command> [<args>] dump [-s <stats socket path>] [-r] [-f <fields>] has is been removed ? do you need to specify something additional now ??? rgds Matt

3 2

James spooling messages problem
by Adam Raszkiewicz 16 Aug '19

16 Aug '19

For some reason my James setup is creating some kind of spooling queue of sending messages. It will queue 3 of them and then forth “sent” message will make 1st one to be sent outside. Any thoughts? Thanks -- Adam

1 0

SASL: encoded packet size too big
by Eugene 16 Aug '19

16 Aug '19

Hello! Dovecot uses it's own SASL implementation, doesn't it? Aug 14 23:45:23 example.com auth[10428]: GSSAPI client step 1 Aug 14 23:45:23 example.com auth[10428]: encoded packet size too big (813804546 > 65536) Aug 14 23:45:23 example.com dovecot[10085]: auth-worker(10428): Error: LDAP: Can't connect to server: ldap://ipa2.example.com Aug 14 23:45:23 example.com dovecot[10085]: auth: Error: auth worker: Aborted USER request for eugene: Lookup timed out Aug 14 23:45:23 example.com dovecot[10085]: imap: Error: auth-master: login: request [3847225345]: Login auth request failed: Internal auth failure (auth connected 60000 msecs ago, request took 60000 msecs, client-pid=10362 client-id=1) Looks like cyrus-sasl encountered same problem earlier. https://lists.andrew.cmu.edu/pipermail/cyrus-sasl/2017-March/003001.html I never have such an issue with ldapsearch. So, I assume there is a similar problem in Dovecot SASL implementation. -- Eugene Bright IT engineer Tel: + 79257289622

3 6

Re: Dovecot - Microsoft Azure AD
by Aki Tuomi 15 Aug '19

15 Aug '19

Hi! Dovecot supports Lua userdb, which can be used to implement custom user databases, maybe this might work for you? See https://doc.dovecot.org/configuration_manual/authentication/lua_based_authe… for more details. Aki > On 15/08/2019 12:16 Lennart Boettcher <lennart.boettcher(a)secpoint.onmicrosoft.com> wrote: > > > Hello, > > Thank you for the quick reply. > > > > > > I have expressed myself wrongly. Our idea was to use the Azure-AD as userdb by doing the user lookup with the help of Microsoft's Graph API. OAuth2 would then of course only be the authorization procedure to access the user accounts using the Graph API. > > > > > > One would then implement a graph-userdb and no oauth-userdb. OAuth is, as you correctly mentioned, only an authorization mechanism. > > > > > > Here is a link to the GraphAPI: https://docs.microsoft.com/de-de/graph/api/overview?view=graph-rest-1.0 > > > And here is another link to the Graph Explorer, with which you can see how the GraphAPI works: https://developer.microsoft.com/en-us/graph/graph-explorer > > > > > We already use this procedure for the passdb lookup and it works very well. > > > > > Greetings > > Lennart Boettcher > > > > > ------------------------------ > > From: Aki Tuomi <aki.tuomi(a)open-xchange.com> > Sent: 14 August 2019 14:57 > To: Lennart Boettcher <lennart.boettcher(a)secpoint.onmicrosoft.com>; Lennart Boettcher via dovecot <dovecot(a)dovecot.org> > Subject: Re: Dovecot - Microsoft Azure AD > > > > On 14/08/2019 15:36 Lennart Boettcher via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > Hello, > > > > I am currently trying to connect my Dovecot mail server to Microsoft's Azure-AD and use it as password and user database. I am using version 2.3.7.1. > > > > > > > > > > > > Using the Azure-AD as passdb already works. In this context I noticed that the scope implementation is not yet merged. > > > > > > > > > > > > Since I haven't found any hints for an OAuth2 userdb implementation yet, I wanted to ask if there are any plans for an implementation. > > > > > > > > > > Greetings > > > > Lennart Boettcher > > > > > > > > > > Dovecot 2.3 supports oauth2. I don't know how "oauth2 user database" would work, since oauth2 is an authentication mechanism. I suggest you use LDAP or static userdb, or set mail_* settings for user settings. > > Aki >

1 0

Dovecot not responding to external clients
by Edwin Humphries 15 Aug '19

15 Aug '19

Hi all I have Dovecot on my Linux-Mint workstation, running IMAP for the Thunderbird client thereon, but previously also for my phone and laptop clients as well. Since upgrading to Linux-Mint 19, however, although the Thunderbird client on the workstation works just fine, the other clients can't connect. I've disabled the firewall, and nmap shows the ports open, so it seems like some setting in the server. Can anyone help? > Dovecot version is 2.2.33.2 (d6601f4ec) > > dovecot -n output: > > # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.21 (92477967) > # OS: Linux 4.15.0-55-generic x86_64 Linux Mint 19.1 Tessa > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_privileged_group = mail > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > driver = pam > } > protocols = " imap" > ssl_cert = </etc/dovecot/private/dovecot.pem > ssl_client_ca_dir = /etc/ssl/certs > ssl_key = # hidden, use -P to show it > userdb { > driver = passwd > } > -- Regards, Edwin Humphries

2 2

Autoexpunge not working for Junk?
by Amir Caspi 14 Aug '19

14 Aug '19

Hi all, I set up dovecot a couple of months ago and am having trouble getting autoexpunge=30d to work on my Trash and Junk mailboxes. Not sure why not because I'm not getting error messages in my log. Running "doveadm search -u <user> mailbox Junk savedbefore 30d" shows me many messages (I've got messages back to mid-May, and a couple of other users have them back to early April, although if this setting were working, there should be nothing earlier than June 24). Running a manual doveadm expunge works fine... it's just autoexpunge that seems to not be running at all. I'm using sendmail as the MTA and procmail as the LDA, so dovecot is running purely for IMAP/POP service. Any help is much appreciated. Thanks! doveconf -n: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.21.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: REDACTED auth_username_format = %Ln first_valid_uid = 1000 mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mailbox_list_index = yes mbox_write_locks = fcntl namespace compat1 { alias_for = hidden = yes list = no location = prefix = mail/ separator = / } namespace compat2 { alias_for = hidden = yes list = no location = prefix = ~/mail/ separator = / } namespace compat3 { alias_for = hidden = yes list = no location = prefix = ~%u/mail/ separator = / } namespace inbox { inbox = yes location = mailbox Archive { special_use = \Archive } mailbox "Deleted Messages" { autoexpunge = 30 days special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { autoexpunge = 30 days special_use = \Junk } mailbox "Junk E-mail" { autoexpunge = 30 days special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { autoexpunge = 30 days special_use = \Junk } mailbox Trash { autoexpunge = 30 days special_use = \Trash } prefix = separator = / } passdb { driver = pam } pop3_uidl_format = %08Xv%08Xu ssl_cert = # REDACTED ssl_cipher_list = # REDACTED ssl_dh_parameters_length = # REDACTED ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { driver = passwd } --- Amir

7 14

segmentation fault in fs_list_get_path
by David M. Johnson 14 Aug '19

14 Aug '19

Hi, There seems to be a straightforward bug in src/lib-storage/list/mailbox-list-fs.c:79. set->index_dir is unchecked prior to dereferencing (unlike on line 126 in the same file, where it is properly checked). This manifested on a FreeBSD server running dovecot 2.3.6 when clients tried to retrieve mail with subscriptions like `~/bar/baz`. This caused the `imap` child to crash, e.g. (slightly anonymized) Core was generated by `imap: [foo w.x.y.z EXAMINE]'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x0000000011593af4 in fs_list_get_path (_list=0x12444848, name=0x12416880 "/home/foo/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe150) at mailbox-list-fs.c:79 79 *set->index_dir == '\0') (gdb) bt #0 0x0000000011593af4 in fs_list_get_path (_list=0x12444848, name=0x12416880 "/home/foo/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe150) at mailbox-list-fs.c:79 #1 0x0000000011559dd0 in mbox_list_get_path (list=0x12444848, name=0x124da410 "~/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe2c8) at mbox-storage.c:96 #2 0x000000001150ed0b in mailbox_list_get_path (list=0x12444848, name=0x124da410 "~/bar/baz", type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe2c8) at mailbox-list.c:1387 #3 0x00000000114f8aaf in get_path_to (box=0x124da048, type=MAILBOX_LIST_PATH_TYPE_INDEX, internal_path=0x124da250, path_r=0x7fffffffe2c8) at mail-storage.c:2662 #4 0x00000000114f89e9 in mailbox_get_path_to (box=0x124da048, type=MAILBOX_LIST_PATH_TYPE_INDEX, path_r=0x7fffffffe2c8) at mail-storage.c:2678 #5 0x00000000114f928c in mailbox_create_missing_dir (box=0x124da048, type=MAILBOX_LIST_PATH_TYPE_INDEX) at mail-storage.c:2826 #6 0x00000000115cf049 in index_storage_mailbox_alloc_index (box=0x124da048) at index-storage.c:243 #7 0x00000000115cf43f in index_storage_mailbox_open (box=0x124da048, move_to_memory=false) at index-storage.c:297 #8 0x000000001155a715 in mbox_mailbox_open_finish (mbox=0x124da048, move_to_memory=false) at mbox-storage.c:413 #9 0x000000001155a94b in mbox_mailbox_open_existing (mbox=0x124da048) at mbox-storage.c:452 #10 0x0000000011559309 in mbox_mailbox_open (box=0x124da048) at mbox-storage.c:489 #11 0x00000000115a446e in mailbox_list_index_open_mailbox (box=0x124da048) at mailbox-list-index.c:720 #12 0x00000000114f43ed in mailbox_open_full (box=0x124da048, input=0x0) at mail-storage.c:1294 #13 0x00000000114f4117 in mailbox_open (box=0x124da048) at mail-storage.c:1350 #14 0x000000000103d788 in select_open (ctx=0x12443228, mailbox=0x12416810 "~/bar/baz", readonly=true) at cmd-select.c:287 #15 0x000000000103d307 in cmd_select_full (cmd=0x12443048, readonly=true) at cmd-select.c:415 #16 0x0000000001034afa in cmd_examine (cmd=0x12443048) at cmd-examine.c:8 #17 0x000000000104a520 in command_exec (cmd=0x12443048) at imap-commands.c:201 #18 0x000000000104804e in client_command_input (cmd=0x12443048) at imap-client.c:1164 #19 0x00000000010483cf in client_command_input (cmd=0x12443048) at imap-client.c:1227 #20 0x00000000010469fc in client_handle_next_command (client=0x12442848, remove_io_r=0x7fffffffe8c7) at imap-client.c:1269 #21 0x00000000010463c0 in client_handle_input (client=0x12442848) at imap-client.c:1283 #22 0x0000000001044134 in client_input (client=0x12442848) at imap-client.c:1329 #23 0x0000000011b6bd1c in io_loop_call_io (io=0x1244f240) at ioloop.c:703 #24 0x0000000011b6fc08 in io_loop_handler_run_internal (ioloop=0x1242b0a0) at ioloop-kqueue.c:160 #25 0x0000000011b6c37e in io_loop_handler_run (ioloop=0x1242b0a0) at ioloop.c:755 #26 0x0000000011b6c146 in io_loop_run (ioloop=0x1242b0a0) at ioloop.c:728 #27 0x0000000011a87dcb in master_service_run (service=0x12436000, callback=0x1060080 <client_connected>) at master-service.c:781 #28 0x000000000105f870 in main (argc=1, argv=0x7fffffffeb20) at main.c:523 (gdb) p set->index_dir $3 = 0x0 The following one-liner fixes the immediate problem (although I didn't look closely to see what set->index_dir means in the context of MAILBOX_LIST_PATH_TYPE_INDEX): --- src/lib-storage/list/mailbox-list-fs.c~ 2019-04-30 06:25:06.000000000 -0600 +++ src/lib-storage/list/mailbox-list-fs.c 2019-08-02 16:23:57.254087000 -0600 @@ -76,6 +76,7 @@ if (mailbox_list_try_get_absolute_path(_list, &name)) { if (type == MAILBOX_LIST_PATH_TYPE_INDEX && + set->index_dir != NULL && *set->index_dir == '\0') return 0; *path_r = name; David

2 1

auth module logging
by AP 14 Aug '19

14 Aug '19

Hi, As per my discussion with cmouse on irc, I'm currently just using dovecot for its auth mechanism, etc capabilities (alas, Exim does not support argon directly). Tis a fun little side project. :) I have the config pared down as far as I think is reasonable (see below) and all is working well except that dovecot is very silent. Errors hit the logs but I would appreciate seeing successful auths happen for the additional piece of mind. Cmouse and I couldn't find a way to do it on irc and it appears that the capability is missing. Successul /logins/ can be logged but auths, by themselves, cannot. I would appreciate if the ability was added. Dovecot 2.3.7.1 is in use. Thanks # Pigeonhole version 0.5.7.1 (db5c74be) # OS: Linux 5.2.4 x86_64 Debian 10.0 # Hostname: bob auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1:16 log_timestamp = "%Y-%m-%d %H:%M:%S %z " mail_log_prefix = "%{pid}<%{session}> %{service}(%{user}): " passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } service anvil { unix_listener anvil-auth-penalty { mode = 00 } } service auth-worker { user = $default_internal_user } service auth { unix_listener auth-client { group = Debian-exim mode = 0660 } } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } AP

3 3

dovecot-uidlist invalid data
by Király Balázs 14 Aug '19

14 Aug '19

Hi! I'm struggling with the following error: Aug 4 21:32:00 mx02 dovecot: imap(xxx(a)xxx.tld)<17693><w14vpU+PCCSwPxpo>: Error: Mailbox INBOX: Broken file /home/vmail/xxx.tld/xxx/dovecot-uidlist line 6246: Invalid data: Aug 4 21:49:22 mx02 dovecot: imap(xxx(a)xxx.tld)<21879><YNhW40+PLoBOGLlU>: Error: Mailbox INBOX: Broken file /home/vmail/xxx.tld/xxx/dovecot-uidlist line 6249: Invalid data: It seems the first part is not incremented properly and sometimes it has a jump in it, like the line 6246: 18810 :1564935891.M816284P8904.mx01.m.ininet.hu,S=12145,W=12409 18812 :1564947092.M542714P2651.mx01.m.ininet.hu,S=12275,W=12517 Even though I deleted all the dovecot* files the error comes back. I have the maildirs through NFS, but I set to store the INDEX files locally. I have failover configuration, that means when one of my servers die another one takes the active IP and start to serve dovecot users, so its always one server that serves all the users, no load balacing. Any idea what am I missing or how could I fixed this problem? uname -a Linux mx02.m.mydomain.tld 4.19.0-5-amd64 #1 SMP Debian 4.19.37-5 (2019-06-19) x86_64 GNU/Linux dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-5-amd64 x86_64 Debian 10.0 nfs # Hostname: mx02.m.mydomain.tld auth_mechanisms = plain login auth_verbose = yes default_client_limit = 10000 default_process_limit = 1000 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no mail_fsync = always mail_location = maildir:/home/vmail/%d/%n:INDEX=/var/indexes/%d/%n mail_plugins = " quota" mail_privileged_group = mail mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota sieve = /home/vmail/default.sieve sieve_default = /home/vmail/default.sieve sieve_extensions = +editheader sieve_global_dir = /var/lib/dovecot/sieve } protocols = " imap pop3" service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = $default_internal_user } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } service imap { process_limit = 1024 } service pop3 { process_limit = 1024 } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert = </etc/postfix/ssl/mail.mydomain.tld-dovecot.crt ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota sieve" } protocol imap { mail_max_userip_connections = 20 mail_plugins = " quota quota imap_quota" } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = " quota quota" } Thanks, Balazs

2 1

Emails not visible after renaming folders
by Aleksandr 14 Aug '19

14 Aug '19

Hello, I have strange problem with "losing" emails after rename mail folder(s) (via imap client: thunderbird, roundcude, etc..) How to reproduce: 1. Create some folder name, like TEST 2. Create sub-folder under TEST (like SUBTEST) Structure: TEST |--SUBTEST # doveadm mailbox list -u postmaster@testmailbox Spam Trash Sent Drafts INBOX TEST TEST/SUBTEST 3. Move (or copy) mails from INBOX to SUBTEST (all looks fine, and mails visible under SUBTEST) 4. Rename TEST folder to any new name, NEWTEST Let`s try to view mails in mail client in NEWTEST-SUBTEST, folder have no emails :( mailsrv# doveadm -f table mailbox status -u postmaster@testmailbox "messages vsize" NEWTEST* mailbox messages vsize NEWTEST 0 0 NEWTEST/SUBTEST 0 0 If doveadm force-resync postmaster@testmailbox, mails will be visible in INBOX mailsrv# doveadm -f table mailbox status -u postmaster@testmailbox "messages vsize" INBOX* mailbox messages vsize INBOX 2 28 Dovecot installation: CentOS x86_64 Linux 7.5.1804 Storage: HDD Local Partition - XFS filesystem / multi-dbox (mdbox) as mail_storage (this problem is not reproduced with the settings as Maildir storage !) somthing wrong with mapping indices. ---- [start] ---- # dovecot -n # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 CentOS Linux release 7.5.1804 (Core) # Hostname: xxxx auth_mechanisms = plain login digest-md5 cram-md5 base_dir = /var/run/dovecot/ default_client_limit = 20000 default_login_user = dovecot default_process_limit = 10000 dict { quota = redis:host=127.0.0.1:prefix=user/:timeout_msecs=1000 } disable_plaintext_auth = no first_valid_gid = 90 first_valid_uid = 90 imapc_features = rfc822.size fetch-headers imapc_host = xxxx imapc_user = %u lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = xxxx. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c login_trusted_networks = 10.0.1.0/24 mail_access_groups = mail mail_debug = yes mail_fsync = never mail_gid = 97 mail_location = mdbox:~/mail/mailboxes:FULLDIRNAME=mBoX-MeSsAgEs:INDEX=~/mail/index:CONTROL=~/mail/control:INBOX=~/mail/mailboxes/inbox mail_log_prefix = "%{session} %Us(%u): " mail_max_lock_timeout = 30 secs mail_plugins = quota zlib mail_prefetch_count = 20 mail_privileged_group = mail mail_uid = 97 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader mbox_lock_timeout = 30 secs mbox_very_dirty_syncs = yes mbox_write_locks = fcntl namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { cgroup_basedir = /usr/sys/cgroup hostingAccount = default quota = dict:User quota::proxy::quota quota_grace = 0%% quota_over_flag_value = TRUE quota_over_script = account-quota mismatch %u quota_rule = *:storage=4T quota_warning = storage=101%% account-quota block %u quota_warning1 = -storage=100%% account-quota unblock %u sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /home/sieve/%d/ sieve_extensions = +editheader +vacation-seconds -body sieve_max_redirects = 200 sieve_vacation_min_period = 1s trash_folder = Trash zlib_save = lz4 zlib_save_level = 6 } pop3_delete_type = expunge protocols = imap pop3 sieve service account-quota { executable = script /usr/local/scripts/account-quota unix_listener account-quota { mode = 0666 user = dovecot } user = dovecot } service anvil { chroot = /var/empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = $default_internal_user idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = $default_internal_user mode = 0660 user = dovecot } unix_listener anvil { group = $default_internal_user mode = 0660 user = dovecot } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = /var/empty user = $default_internal_user } service auth { chroot = /var/empty executable = /usr/libexec/dovecot/auth unix_listener auth-client { mode = 0660 } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0777 } user = $default_internal_user } service dict { executable = /usr/libexec/dovecot/dict unix_listener dict { mode = 0600 user = $default_internal_user } } service imap-login { executable = /usr/libexec/dovecot/imap-login inet_listener imap { address = xxxxxxx port = 143 } process_limit = 10000 process_min_avail = 1 service_count = 0 user = dovecot } service imap { executable = /usr/libexec/dovecot/imap process_limit = 18000 process_min_avail = 1 service_count = 0 vsz_limit = 256 M } service lmtp { process_limit = 10 } service pop3-login { executable = /usr/libexec/dovecot/pop3-login inet_listener pop3 { address = xxxxxx port = 110 } process_limit = 1000 process_min_avail = 1 service_count = 0 user = dovecot vsz_limit = 256 M } service pop3 { executable = /usr/libexec/dovecot/pop3 process_limit = 1000 process_min_avail = 1 service_count = 0 } service send-warning { executable = script /usr/local/scripts/send-warning unix_listener send-warning { mode = 0666 user = dovecot } user = dovecot } shutdown_clients = no ssl_cert = </etc/dovecot/xxxx.crt ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } verbose_proctitle = yes protocol lda { mail_plugins = sieve quota cgroups zlib } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 50 mail_plugins = quota imap_quota mail_log notify cgroups zlib } protocol pop3 { mail_max_userip_connections = 20 mail_plugins = quota cgroups zlib pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = no pop3_no_flag_updates = yes pop3_uidl_format = %08Xu%08Xv } ---- [end] ---- Regards, Aleksandr.

3 4

Auth driver
by Riccardo Paolo Bestetti 14 Aug '19

14 Aug '19

(resending to the list; I apologize, I'm not using my usual email client) Hello, That's actually great news. The perspective of working in-tree didn't make me particularly happy. Could you point me to any documentation or examples? While I can find many plugins in the repo and around the Internet, I could find none which add authdb drivers. Best Regards, Riccardo P. Bestetti Da: Aki Tuomi <aki.tuomi(a)open-xchange.com> Inviato: venerdì 9 agosto 2019 13:56 A: Riccardo Paolo Bestetti <riccardo.kyogre(a)live.it>; dovecot(a)dovecot.org <dovecot(a)dovecot.org> Oggetto: Re: Auth driver On 9.8.2019 14.51, Riccardo Paolo Bestetti via dovecot wrote: > Greetings! > > I'm planning to implement a new auth driver. It's going to be, in concept, similar to the Lua and CheckPassword drivers, in that it allows an user program to carry out the authentication and user enumeration steps. If you do this, please make it as 3rd party repository. Dovecot auth supports plugins. Aki

3 2

Should dovecot not be using different logging facility and severity levels?
by Marc Roos 14 Aug '19

14 Aug '19

Should dovecot not be using different severity levels like auth.warn? On my system everything goes to loglevel info: lev_info:Aug 9 16:18:24 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<UBXJ2K+PYh68zmjw> lev_info:Aug 9 16:18:29 mail03 dovecot: auth-worker(28656): pam(krinfo,188.206.104.240,<LOLx2K+PYx68zmjw>): unknown user lev_info:Aug 9 16:18:50 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 25 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS: Disconnected, session=<LOLx2K+PYx68zmjw> lev_info:Aug 9 16:18:53 mail03 dovecot: auth-worker(28656): pam(krinfo,188.206.104.240,<qJOm2q+Pax68zmjw>): unknown user lev_info:Aug 9 16:19:01 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 8 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<qJOm2q+Pax68zmjw> lev_info:Aug 9 16:19:13 mail03 dovecot: auth-worker(28656): pam(krinfo,188.206.104.240,<k8/X26+Pch68zmjw>): unknown user lev_info:Aug 9 16:19:15 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<k8/X26+Pch68zmjw> lev_info:Aug 9 16:19:24 mail03 dovecot: auth-worker(28656): pam(krinfo,188.206.104.240,<MjBy3K+Pfh68zmjw>): unknown user lev_info:Aug 9 16:19:26 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<MjBy3K+Pfh68zmjw> lev_info:Aug 9 16:19:27 mail03 dovecot: auth-worker(28656): pam(krinfo,188.206.104.240,<oRmo3K+Pfx68zmjw>): unknown user lev_info:Aug 9 16:19:29 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<oRmo3K+Pfx68zmjw> lev_info:Aug 9 16:19:47 mail03 dovecot: auth-worker(29664): pam(krinfo,188.206.104.240,<14Pb3a+Pih68zmjw>): unknown user lev_info:Aug 9 16:19:49 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<14Pb3a+Pih68zmjw> lev_info:Aug 9 16:19:51 mail03 dovecot: auth-worker(29664): pam(krinfo,188.206.104.240,<99cO3q+Pix68zmjw>): unknown user lev_info:Aug 9 16:19:53 mail03 dovecot: imap-login: Aborted login (auth failed, 1 attempts in 2 secs): user=<xxxxx>, method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, TLS, session=<99cO3q+Pix68zmjw> This is how failed attempts are logged by vsftpd fac_authpriv:Aug 9 16:24:42 web01 vsftpd[7255]: pam_ldap(vsftpd:auth): Authentication failure; user=xxxxx fac_authpriv:Aug 9 16:24:42 web01 vsftpd[7255]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=xxxxx rhost=xxxxx user=xxxxx fac_ftp:Aug 9 16:24:44 web01 vsftpd[7255]: [xxxxx] FAIL LOGIN: Client "x.x.x.x" lev_notice:Aug 9 16:24:42 web01 vsftpd[7255]: pam_ldap(vsftpd:auth): Authentication failure; user=xxxxx lev_notice:Aug 9 16:24:42 web01 vsftpd[7255]: pam_unix(vsftpd:auth): authentication failure; logname= uid=0 euid=0 tty=ftp ruser=xxxxx rhost=xxxxx user=xxxxx lev_warn:Aug 9 16:24:44 web01 vsftpd[7255]: [xxxxx] FAIL LOGIN: Client "x.x.x.x" Using dovecot-2.2.36-3.el7.x86_64 on CentOS7

4 4

ACL ignored for master users
by Philip Iezzi 14 Aug '19

14 Aug '19

Hi there! I can't get ACL working for master users. Login as master user works fine though, and I am able to access any mailbox using the auth_master_user_separator "*", tested e.g. via Python's imaplib: import imaplib imap = imaplib.IMAP4_SSL('imap.example.com') imap.login('foo(a)example.com*admin-acldemo', '**********') ('OK', [b'Logged in']) My /etc/dovecot/dovecot-acl looks like this: * user=admin lr bar(a)example.com user=admin-acldemo lr So, if I didn't misunderstand https://wiki.dovecot.org/Authentication/MasterUsers and https://wiki.dovecot.org/ACL documentation, this should only give "admin" master user access to all mailaccounts and limiting "admin-acldemo" master user to only a single mailaccount. That's what I would like to accomplish. But no matter what I put into dovecot-acl (it could even be empty), master users always have access to all existing mailaccounts. The whole dovecot-acl seems to be ignored and there are no logs pointing to any problem (syntax, access permissions) with that file. In mail.log I am getting a successful login message: dovecot: imap-login: Login: user=<foo(a)example.com>, method=PLAIN, ... So the login with "foo(a)example.com*admin-acldemo" seems to get resolved to a regular "foo(a)example.com" user login. My current setup (relevant config options): # 2.3.7.1 (0152c8b10): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.1 (db5c74be) # OS: Linux 4.15.18-18-pve x86_64 Debian 9.9 auth_master_user_separator = * mail_plugins = quota acl passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } plugin { acl = vfile:/etc/dovecot/dovecot-acl acl_user = %u master_user = %u } protocol lmtp { mail_plugins = quota acl sieve } protocol lda { mail_plugins = quota acl sieve } protocol imap { mail_plugins = quota acl imap_acl imap_quota } On ACL documentation it says: > Note that master users have their own ACLs. They're not the the mailbox owners, so by default they have no permissions to any of the mailboxes and on Authentication/MasterUsers documentation: > If ACL plugin is enabled, the Master user is still subject to ACLs just like any other user, which means that by default the master user has no access to any mailboxes of the user. So it must be somehow possible to limit master users to specific mailbox(es) via ACL. I could allow master users to be able to log in as themselves, by adding two passdb sections (one with `master = yes`, the other `master = no`), but then I see no way to do a user context switch (where ACLs would get respected...) after logging in with "admin". Any help greatly appreciated! Thanks in advance. Cheers, Philip

1 1

Dovecot - Microsoft Azure AD
by Lennart Boettcher 14 Aug '19

14 Aug '19

Hello, I am currently trying to connect my Dovecot mail server to Microsoft's Azure-AD and use it as password and user database. I am using version 2.3.7.1. Using the Azure-AD as passdb already works. In this context I noticed that the scope implementation is not yet merged. Since I haven't found any hints for an OAuth2 userdb implementation yet, I wanted to ask if there are any plans for an implementation. Greetings Lennart Boettcher

2 1

Solr, Dovecot & macOS / iOS
by James Brown 14 Aug '19

14 Aug '19

I’m thinking of getting Solr working with my Dovecot server. Server is new 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of messages. But I’m not sure if: a) it will make enough of a difference and b) does Mail.app and other mail clients on Macs or iOS devices perform searches on their local copy of mail or does it just send a search request to the server? I’m guessing the searches are done locally so no point in Solr? Does anyone know? Thanks, James.

4 5

Dovecot Sieve doesn't move spam to Junk folder in maildir
by Nagy Tibor 13 Aug '19

13 Aug '19

I have an *Ubuntu 16.04* mail server with postfix/spamassassin/*dovecot-2.2.2**2* with virtual mailboxes in maildir format. The whole process is working excellently since a year. Spams are marked with "*****SPAM*****" in subject and with "X-Spam-Flag: YES" in email header. I tried now to add sieve to dovecot to move the SPAM messages into the Junk folder without any success, I don't see any sign that sieve does something at all. The output of "dovecot -n": # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-72-generic x86_64 Ubuntu 16.04.2 LTS ext4 auth_mechanisms = cram-md5 auth_verbose = yes base_dir = /var/run/dovecot/ info_log_path = /var/log/dovecot.info log_path = /var/log/dovecot log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { inbox = yes location = prefix = separator = / } passdb { args = /etc/dovecot/passwd driver = passwd-file } plugin { sieve = /etc/dovecot/sieve/default.sieve } protocols = imap pop3 service auth { executable = /usr/lib/dovecot/auth unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = root } service imap-login { chroot = login executable = /usr/lib/dovecot/imap-login inet_listener imap { port = 0 } user = dovecot } service imap { executable = /usr/lib/dovecot/imap } service pop3-login { chroot = login executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } user = dovecot } service pop3 { executable = /usr/lib/dovecot/pop3 } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/users driver = passwd-file } valid_chroot_dirs = /var/spool/vmail protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = " sieve" } I've added only plugin and protocol lda now. The content of the file default.sieve: require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "Junk"; } The test SPAM message is marked as SPAM, but not moved to the folder Junk. I don't see anything in mail.log that is associated to sieve.: ul 30 20:05:12 zg-3 postfix/smtpd[4446]: connect from smtp4.enternet.hu[62.112.192.37] Jul 30 20:05:13 zg-3 postfix/smtpd[4446]: 3C4CC9F570: client=smtp4.enternet.hu[62.112.192.37] Jul 30 20:05:13 zg-3 postfix/cleanup[4450]: 3C4CC9F570: message-id=<c6b3bca5-ece3-9e69-8e30-28173abc2ec6(a)hu.inter.net> Jul 30 20:05:13 zg-3 postfix/qmgr[1557]: 3C4CC9F570: from=<nagyt(a)hu.inter.net>, size=918, nrcpt=1 (queue active) Jul 30 20:05:13 zg-3 postfix/smtpd[4446]: disconnect from smtp4.enternet.hu[62.112.192.37] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7 Jul 30 20:05:13 zg-3 postfix/pickup[3949]: 5F7FEA3CBC: uid=1003 from=<nagyt(a)hu.inter.net> Jul 30 20:05:13 zg-3 postfix/pipe[4451]: 3C4CC9F570: to=<tibor(a)softxs.ch>, relay=spamassassin, delay=0.15, delays=0.05/0.01/0/0.1, dsn=2.0.0, status=sent (delivered via spamassassin service) Jul 30 20:05:13 zg-3 postfix/qmgr[1557]: 3C4CC9F570: removed Jul 30 20:05:13 zg-3 postfix/cleanup[4450]: 5F7FEA3CBC: message-id=<c6b3bca5-ece3-9e69-8e30-28173abc2ec6(a)hu.inter.net> Jul 30 20:05:13 zg-3 postfix/qmgr[1557]: 5F7FEA3CBC: from=<nagyt(a)hu.inter.net>, size=3304, nrcpt=1 (queue active) Jul 30 20:05:13 zg-3 postfix/virtual[4455]: 5F7FEA3CBC: to=<tibor(a)softxs.ch>, relay=virtual, delay=0.02, delays=0.01/0.01/0/0, dsn=2.0.0, status=sent (delivered to maildir) Jul 30 20:05:13 zg-3 postfix/qmgr[1557]: 5F7FEA3CBC: removed What can be wrong?

2 2

dovecot acl delete messages questions
by hfh＠cndns.com 13 Aug '19

13 Aug '19

Dovecot's acl plugin can prevent deletion of messages,it works very well! I want: Some ips can delete messages, other ips can not delete messages,can anyone help me? hfh(a)cndns.com

1 0

LastLogin update
by Jorge Bastos 12 Aug '19

12 Aug '19

Howdy, I'd like an adive on what to use. I have sql users, and i want to have the lastlogin of them, what is the native option for this: * Using last_login plugin which will give me the last login? * Use postlogin for imap and pop3n which will give me besides lastlogin the IP and protocol? Thanks in advanced,

8 13

ACL ignored for master users
by Philip Iezzi 12 Aug '19

12 Aug '19

Hi there! I can't get ACL working for master users. Login as master user works fine though, and I am able to access any mailbox using the auth_master_user_separator "*", tested e.g. via Python's imaplib: >>> import imaplib >>> imap = imaplib.IMAP4_SSL('imap.example.com') >>> imap.login('foo(a)example.com*admin-acldemo', '**********') ('OK', [b'Logged in']) My /etc/dovecot/dovecot-acl looks like this: * user=admin lr bar(a)example.com user=admin-acldemo lr So, if I didn't misunderstand https://wiki.dovecot.org/Authentication/MasterUsers and https://wiki.dovecot.org/ACL documentation, this should only give "admin" master user access to all mailaccounts and limiting "admin-acldemo" master user to only a single mailaccount. But no matter what I put into dovecot-acl (it could even be empty), master users always have access to all existing mailaccounts. The whole dovecot-acl seems to be ignored and there are no logs pointing to any problem (syntax, access permissions) with that file. In mail.log I am getting a successful login message: dovecot: imap-login: Login: user=<foo(a)example.com>, method=PLAIN, rip=..., lip=..., mpid=42371, TLS, session=<T9GKEuyPKGxRPtGS> My current setup (relevant config options): # 2.3.7.1 (0152c8b10): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7.1 (db5c74be) # OS: Linux 4.15.18-18-pve x86_64 Debian 9.9 auth_master_user_separator = * mail_plugins = quota acl passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } plugin { acl = vfile:/etc/dovecot/dovecot-acl acl_user = %u master_user = %u } protocol lmtp { mail_plugins = quota acl sieve } protocol lda { mail_plugins = quota acl sieve } protocol imap { mail_plugins = quota acl imap_acl imap_quota } On ACL documentation it says: > Note that master users have their own ACLs. They're not the the mailbox owners, so by default they have no permissions to any of the mailboxes and on Authentication/MasterUsers documentation: > If ACL plugin is enabled, the Master user is still subject to ACLs just like any other user, which means that by default the master user has no access to any mailboxes of the user. So it must be somehow possible to limit master users to specific mailbox(es) via ACL. If I disable `master = yes` master users won't be able to login at all. Any help greatly appreciated! Thanks in advance. Cheers, Philip

1 0

dovecot-lmtp and postfix
by Coy Hile 12 Aug '19

12 Aug '19

Hi all, I’ve got postfix setup to use dovecot-lmtp for (virtual) user delivery, and things to users or aliases that Dovecot knows about now get delivered correctly. However, I’ve done something wrong with postfix and/or the LMTP configuration because upon RCPT TO, instead of forwarding the mail off to the configured smartest, somehow dovecot-lmtp is ending up in the mix. (Perhaps this is more a postfix question than a dovecot one, but, I’ll start here.) Relevant postfix config: [root@81716ec5-bca4-6d53-ed81-bd1a55d46b4f /opt/local/etc/postfix]# postconf -n canonical_maps = hash:/opt/local/etc/postfix/canonical command_directory = /opt/local/sbin compatibility_level = 2 daemon_directory = /opt/local/libexec/postfix data_directory = /var/db/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 default_database_type = hash home_mailbox = Maildir/ html_directory = no inet_interfaces = all inet_protocols = all mail_owner = postfix mailbox_size_limit = 102400000 mailq_path = /opt/local/bin/mailq manpage_directory = /opt/local/man message_size_limit = 51200000 meta_directory = /opt/local/libexec/postfix mynetworks = $myhostname, 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12 192.168.0.0/16 newaliases_path = /opt/local/bin/newaliases owner_request_special = no queue_directory = /var/spool/postfix queue_minfree = 76800000 readme_directory = /opt/local/share/doc/postfix recipient_canonical_maps = hash:/opt/local/etc/postfix/recipient_canonical recipient_delimiter = + relayhost = relay.svc.a4b01392-e48d-4cdf-cacf-f850d1800d45.basement.cns.coyhile.com sample_directory = /opt/local/share/examples/postfix sender_canonical_maps = hash:/opt/local/etc/postfix/sender_canonical sendmail_path = /opt/local/sbin/sendmail setgid_group = maildrop shlib_directory = /opt/local/lib/postfix smtpd_banner = $myhostname ESMTP smtpd_client_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_reverse_client_hostname smtpd_data_restrictions = permit_mynetworks, reject_unauth_pipelining, permit_sasl_authenticated smtpd_helo_required = yes smtpd_helo_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_invalid_helo_hostname, reject_non_fqdn_helo_hostname smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unlisted_recipient, reject_non_fqdn_recipient, reject_unauth_destination, reject_unknown_recipient_domain, check_recipient_access hash:/opt/local/etc/postfix/filtered_domains smtpd_sasl_auth_enable = yes smtpd_sasl_path = private/auth smtpd_sasl_type = dovecot smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unlisted_sender, reject_non_fqdn_sender, reject_unknown_sender_domain smtputf8_enable = no strict_rfc821_envelopes = yes unknown_local_recipient_reject_code = 550 virtual_alias_maps = pgsql:/opt/local/etc/postfix/postfix_virtual_aliases.cf virtual_mailbox_domains = pgsql:/opt/local/etc/postfix/postfix_virtual_domains.cf virtual_transport = lmtp:unix:private/dovecot-lmtp [root@81716ec5-bca4-6d53-ed81-bd1a55d46b4f /opt/local/etc/postfix]# And the dovecot config: doveconf -n # 2.3.6 (7eab80676): /opt/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.6 (92dc263a) doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -Pn > dovecot-new.conf doveconf: Warning: Obsolete setting in /opt/local/etc/dovecot/conf.d/10-ssl.conf:80: ssl_dh_parameters_length is no longer needed doveconf: Warning: Obsolete setting in /opt/local/etc/dovecot/conf.d/10-ssl.conf:81: ssl_protocols has been replaced by ssl_min_protocol doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -Pn > dovecot-new.conf doveconf: Warning: Obsolete setting in /opt/local/etc/dovecot/conf.d/10-ssl.conf:80: ssl_dh_parameters_length is no longer needed doveconf: Warning: Obsolete setting in /opt/local/etc/dovecot/conf.d/10-ssl.conf:81: ssl_protocols has been replaced by ssl_min_protocol doveconf: Warning: service auth { client_limit=840 } is lower than required under max. load (1524) # OS: SunOS 5.11 i86pc # Hostname: 81716ec5-bca4-6d53-ed81-bd1a55d46b4f.coyhile.com auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb imap_idle_notify_interval = 29 mins lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_attachment_dir = /var/mail/attachments mail_fsync = never mail_location = mdbox:~/mdbox mail_plugins = " acl" mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /opt/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl_defaults_from_inbox = yes imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_global_extensions = +vnd.dovecot.pipe sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = imap pop3 lmtp submission service auth-worker { user = vmail } service auth { client_limit = 840 inet_listener { port = 2222 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_min_avail = 3 service_count = 0 vsz_limit = 1 G } service imap { process_min_avail = 3 service_count = 256 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } ssl_cert = </opt/local/etc/dovecot/certs/dovecot.pem ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = /opt/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " acl sieve" } protocol lda { mail_plugins = " acl sieve" } protocol imap { mail_max_userip_connections = 50 mail_plugins = " acl imap_acl imap_sieve” } And in syslog I see: 2019-08-11T15:37:57+00:00 81716ec5-bca4-6d53-ed81-bd1a55d46b4f postfix/smtpd[56103]: [ID 197553 mail.info] 12327F13D: client=ns1.coyhile.com[172.17.64.25] 2019-08-11T15:38:40+00:00 81716ec5-bca4-6d53-ed81-bd1a55d46b4f postfix/cleanup[56365]: [ID 197553 mail.info] 12327F13D: message-id=<> 2019-08-11T15:38:40+00:00 81716ec5-bca4-6d53-ed81-bd1a55d46b4f postfix/qmgr[56018]: [ID 197553 mail.info] 12327F13D: from=<coy.hile(a)coyhile.com>, size=354, nrcpt=1 (queue active) 2019-08-11T15:38:40+00:00 81716ec5-bca4-6d53-ed81-bd1a55d46b4f dovecot: [ID 702911 mail.info] lmtp(50179): Connect from local 2019-08-11T15:38:40+00:00 81716ec5-bca4-6d53-ed81-bd1a55d46b4f postfix/lmtp[56821]: [ID 197553 mail.info] 12327F13D: to=<coyhile(a)gmail.com>, relay=81716ec5-bca4-6d53-ed81-bd1a55d46b4f.localdomain[private/dovecot-lmtp], delay=55, delays=55/0.01/0/0.03, dsn=5.1.1, status=bounced (host 81716ec5-bca4-6d53-ed81-bd1a55d46b4f.localdomain[private/dovecot-lmtp] said: 550 5.1.1 <coyhile(a)gmail.com> User doesn't exist: coyhile(a)gmail.com (in reply to RCPT TO command)) Given both of those configurations, shouldn’t it see that gmail address and send it off to the relay host? -Coy

2 2

Gmail to Dovecot migration
by Yuri 12 Aug '19

12 Aug '19

Hi everyone, I'm trying to migrate my gmail to dovecot following instructions at https://wiki.dovecot.org/Migration/Gmail Using -a 'virtual/All' option does not seem to work because the gmail folder with that name does not exist, but -a '[Gmail]/All Mail' seem to work. I'm using the following command: sudo doveadm \ -o imapc_host=imap.gmail.com \ -o imapc_user=user(a)gmail.com \ -o imapc_password=gmailpassword \ -o imapc_features="rfc822.size gmail-migration" \ -o imapc_port=993 \ -o imapc_ssl=imaps \ -o ssl_client_ca_dir=/etc/ssl/certs \ -o imapc_ssl_verify=yes \ backup -a '[Gmail]/All Mail' -O '-$GmailHaveLabels' -R -u user(a)exmaple.com imapc: The problem is this command only migrates approximately 25% of my emails by size. Google Takeout exports 4 times more data in mbox format, the size of exported mboxes aligns with what Google tells me a the bottom of gmail settings page. I experimented with other options of 'doveadm backup' command, but they all resulted in less data being migrated. The above options are the best I was able to achive so far. Another option I was considering is to convert Google Takeout mboxes to dovecot format. I tried using 'doveadm import' command, but got confused with various user and search query options of the command. The best I was able to achive was to output a folder under /Takeout which mimics my mbox structure, but with no imported messages. A third option I'm considering is to convert Google Takeout mboxes using mb2md command. Conversion seems to work fine, but the output is in the traditional (dotted) format, and for historical reasons I use 'LAYOUT=fs' option. So I need to do some extra work to convert folder names. And even if I do so, I'm still not sure if I can import Maildir to dovecot because dovecot's Maildir has other information, such as indices, logs, etc., not just email folders. I'll keep investigating the second and the third option because it seems that Google Takeout gives me all my emails, but I'm not sure if those are the right paths. I wanted to ask the experts about the best option to migrate gmail to dovecot. Has anyone being able to successfully migrate all their emails from gmail and what steps have been used for that? My Dovecot version: 2.3.4.1 Thank you, Yuri

1 1

What does `iterate_query` for SQL want as output?
by Coy Hile 10 Aug '19

10 Aug '19

Hi all, In an earlier thread, https://dovecot.org/pipermail/dovecot/2019-August/116694.html I got a lot of useful help about migration. On my older host, everything was static; on the newer host, I’m storing user information in Postgres. usernames are of the form <login@REALM>, say ‘hile(a)coyhile.xn--com-to0a as basically a Kerberos principal, and authentication and individual lookups work. My `users` table looks thus: mail=> \d users Table "public.users" Column | Type | Modifiers ----------+------+----------- username | text | not null domain | text | not null password | text | not null mail=> and contains, as an example: username | domain | password ------------------+-------------+------------------------------------------------------------------------------------------------------- hile(a)coyhile.com | coyhile.com | [REDACTED] (1 row) Naively, I’d expect something this to work for the iteration query: iterate_query = SELECT username, domain FROM users But, when I do that, I end up doveadm backup -D -A -R -f ssh -i id_rsa.dsync imap01.coyhile.com /opt/local/bin/doveadm dsync-server -A doveadm(hile@coyhile.com@coyhile.com): Info: User no longer exists, skipping [root@81716ec5-bca4-6d53-ed81-bd1a55d46b4f /tmp]# Note the extra “(a)coyhile.xn--com-9o0a in there. Thanks, — Coy Hile coy.hile(a)coyhile.com

2 6

Auth driver
by Riccardo Paolo Bestetti 09 Aug '19

09 Aug '19

Greetings! I'm planning to implement a new auth driver. It's going to be, in concept, similar to the Lua and CheckPassword drivers, in that it allows an user program to carry out the authentication and user enumeration steps. The rationale is that such a solution would provide better decoupling between Dovecot and the user authentication and enumeration logic. Existing integrated logic (auth/ directory in the source tree) is, even encompassing a great number of common use cases, ultimately inflexible. Writing custom drivers implementing logic, while being a solution to the inflexibility problem, is clearly not what drivers were engineered for: for one they don't support dynamic loading, making it necessary to recompile Dovecot to make changes, which is unacceptable in many situations, e.g. when using a packaged or containerized version of the software. In this email, I intend to explain what characteristics my driver will have and why it solves the proposed problem better than existing code (such as CheckPassword, Lua and proxy), in the hope to get useful feedback and I will pose some technical questions to which I could not find definitive answers by myself yet. These are going to be the characteristics of my driver (with particular emphasis to the differences to the existing drivers): - It is going to be a completely generic mechanism exposing all funcionality of Dovecot's interface, both password check and enumeration, and will fully support authentication methods with continuations. - It is going to be performant, communicating with external programs through sockets (unix or inet). - It is going to communicate using a simple, dynamic-schema, well-understood protocol, which is probably going to be JSON, being very easy to generate and parse. This is why the existing drivers do not already provide what I described: - CheckPassword is heavy (fork & 2*exec), and it's generally a mess, having initially been provided as a compatibility feature and having then been hacked to support additional functionality; it's architecturally unfeasable to support continuations with it and anyway any further extension will probably make it explode :) - Lua works but it's limited, in that it dictates a language and also dictates that the auth logic should run on the same server as Dovecot (and under Dovecot's control). It also doesn't seem to support plain password checks, but only user-password enumeration, making it non-generic (at least this is what can be inferred from the documentation and examples.) - Other possible solutions, such as (mis)using the proxy driver to communicate with an external program are not well documented and nothing guarantees the proxy protocol will not change anyway, being an internal feature. And now, the technical questions. I know these could be answered by reading the code, which is something I'm doing, but I'm finding some architectural details quite difficult to grasp due to their complexity: - From what I gather from [1], auth-worker processes only process a single request before being killed from auth. Would setting service_count to something different cause auth-workers to be reused or would it just leak the auth-workers? In the first case, is the preinit of the auth driver repeated, or is the existing instance of the auth driver reused? You could also point me to the relevant code sections which start, keep track of and initialize auth-workers. - How does a driver specify what to run in an auth-worker as opposed to in the auth process? (This is strongly related to the first question, and maybe it is more general, so you may want to answer this one first.) Please, feel free to only answer parts of my email. Any input on any of the matters is very welcome. Best regards, Riccardo P. Bestetti [1] https://wiki.dovecot.org/Services#auth-worker

2 1

Re: Upgrading to v2.3.X breaks ssl san?
by Aki Tuomi 09 Aug '19

09 Aug '19

> On 07/08/2019 14:28 telsch <telsch(a)gmx.de> wrote: > > > with v2.2.34 i can use: > > ssl_ca = </etc/ssl/ca-bundle.pem > ssl_cert = </etc/ssl-imap.pem > > after upgrade to v2.3.X it doesn't work like before. > > it's working if i manual cat ca-bundle.pem and ssl-imap.pem into one > file and using only: > > ssl_cert = </etc/ssl-imap.pem > > i thought ssl_ca is where to put the intermediate cert? (Sorry for duplicate mail, keyboard acted up...) No, that has always been a mistake and it was fixed in 2.3. Our SSL pages in documentation & wiki have always recommended concatenating the intermediates with the cert. Aki

3 3

IMAP folders showing no messages in Office 365 Outlook
by Andrew Bernard 08 Aug '19

08 Aug '19

I'm new to Dovecot. Using Dovecot IMAP with Postfix, everything works fine using Thunderbird. But with the same setup, no messages show in IMAP folders for user accounts created in Outlook, accessing exactly the same server as the accounts in Thunderbird. Dovecot version 2.3.7.1. Is there any configuration required that is special to support Office 365 Outlook clients? How to debug this? Andrew

3 3

Re: How to make the client tool only display emails within 30 days?
by hfh＠cndns.com 08 Aug '19

08 Aug '19

Thank you so much, I tried it！ hfh(a)cndns.com 发件人： Aki Tuomi via dovecot 发送时间： 2019-08-08 15:47 收件人： hfh(a)cndns.com; dovecot 主题： Re: How to make the client tool only display emails within 30 days? On 8.8.2019 10.46, hfh--- via dovecot wrote: doveadm search -u matt mailbox "*" savedbefore 30d Dovecot's configuration or plugin can do similar effects like the above？ Outlook uses pop or imap to receive mail within 30 days. Virtual plugin can be used to do these kind of configurations. Aki

1 0

How to make the client tool only display emails within 30 days?
by hfh＠cndns.com 08 Aug '19

08 Aug '19

doveadm search -u matt mailbox "*" savedbefore 30d Dovecot's configuration or plugin can do similar effects like the above？ Outlook uses pop or imap to receive mail within 30 days.

2 1

How to make the client tool only display emails within 30 days?
by hfh＠cndns.com 08 Aug '19

08 Aug '19

How to make the client tool only display emails within 30 days? thanks！！！ hfh(a)cndns.com

2 1

Upgrading to v2.3.X breaks ssl san?
by telsch 07 Aug '19

07 Aug '19

3 5

Problem Solr and centos 7
by HTMLServices.it 07 Aug '19

07 Aug '19

Hi everyone Given that I am not an expert, I am doing tests with Solr, I installed following the guide but I have no benefits on the search, the search on the body on 28000 mails takes a few minutes and then goes to timeout. I had installed version 8.2.0 of solr then I thought that there is something not compatible with centos 7 so I installed version 7.7.0 as an example on the guide https://wiki2.dovecot.org/Plugins/FTS/Solr. I did several tests but I can't get it to work, this is the test server link: http://5.39.2.59:8987/solr/#/ I only changed the door because the 8983 is busy even making a query from the panel it seems to me not working .... thanks to all Bic

2 3

auth-policy crashing
by James 07 Aug '19

07 Aug '19

My auth process is dumping core. This happens several times per day but dovecot can operate normally for hours between errors. The crash occurs in src/auth/auth-policy.c, line 356: t@1 (l@1) program terminated by signal SEGV (no mapping at the fault address) Current function is auth_policy_parse_response 356 context->request->policy_refusal = FALSE; ...context->request is null. Add markers to the code... } if (context->request == NULL) fprintf(stderr, "2222222222222222222222222222 context->request == NULL\n"); i_stream_unref(&context->payload); if (context->request == NULL) fprintf(stderr, "1111111111111111111111111111 context->request == NULL\n"); if (context->parse_error) { context->result = (context->set->policy_reject_on_fail ? -1 : 0); } if (context->request == NULL) fprintf(stderr, "AAAAAAAAAAAAAAAAAAAAAAAAAAAA context->request == NULL\n"); context->request->policy_refusal = FALSE; if (context->result < 0) { ...gives at the time of the crash... Aug 1 14:25:44 mailhost dovecot: [ID 702911 mail.error] auth: Error: 1111111111111111111111111111 context->request == NULL Aug 1 14:25:44 mailhost dovecot: [ID 702911 mail.error] auth: Error: AAAAAAAAAAAAAAAAAAAAAAAAAAAA context->request == NULL ...so context->result is not null before the call (no 222) to i_stream_unref but is after. dovecot.conf has: auth_policy_server_url = http://policyserver.lan/ auth_policy_server_timeout_msecs = 3000 auth_policy_hash_nonce = Ohr9phaeSeip2Pahaez2raiGohxoo5Ia auth_policy_request_attributes = remote=%{rip} auth_policy_check_before_auth = yes auth_policy_check_after_auth = yes auth_policy_report_after_auth = yes To simplify the problem I used a dummy policy server, in nginx.conf: location / { default_type application/json; return 200 "{\"status\":0,\"msg\":\"accepted\"}"; } however no matter what rubbish a policy server sends back it should not cause dovecot to crash. I've tried 32 and 64 bit and two compilers (gcc and SunStudio), all result in crashes. Adding: keepalive_timeout 0; ...to nginx.conf appears to reduce the crashes. It happens with a variety of users and with debug output I see no pattern. James.

2 6

[BUG?] Double quota calulation when special folder is present
by Mark Moseley 07 Aug '19

07 Aug '19

Just hoping to get some dev eyes on this. I'm incredibly reluctant to throw the word 'bug' around (since 99 times out of 100, it's not -- it's almost always the config), but I can't think of any way that this could be a config issue, esp when the pre-2.2.34 version works as expected. I noticed during troubleshooting that dovecot errors out if I try to create a subfolder called 'INBOX' but it'll happily create a subfolder called INBOX.SomethingElse (i.e. a folder called INBOX.INBOX.SomethingElse - resulting in a directory called .INBOX.SomethingElse on the filesystem, and leading to the problem described below). Is that sub-subfolder creation (where the top level subfolder matches the namespace name) supposed to be allowed? It seems odd that 'INBOX' (as a subfolder of INBOX) would be blocked but INBOX.SomethingElse (as a subfolder of INBOX) would be allowed. I'd expect INBOX.SomethingElse (i.e. INBOX.INBOX.SomethingElse) would be blocked as well. On Wed, Mar 13, 2019 at 4:46 AM Bernd Wurst via dovecot <dovecot(a)dovecot.org> wrote: > Hello, > > we're operating dovecot on a small server. Some years ago, we migrated > from courier IMAP to dovecot. Therefore, we defined our default > Namespace "inbox" with prefix "INBOX." to have this compatible. I found > this in some migration docs those days. Generally, everything worked as > expected. > > Our only namespace is configured like this: > > namespace inbox { > separator = . > prefix = INBOX. > inbox = yes > } > > Regularly, there is no folder named INBOX or .INBOX in the file system, > I suppose this is correct. But I found a special corner case today when > it comes to quota calculation. > > When - for whatever reason - a folder .INBOX.foo (for arbitrary values > of foo) exists, the whole mailbox is counted twice in quota > recalculation. Just creating .INBOX does nothing but a subfolder > triggers the problem. > > This is my shell view (replaced username and file path and deleted > unnecessary debug output) > > $ cat maildirsize > 268435456S > 14697 17 > $ maildirmake .INBOX.foo > $ sudo doveadm -D quota recalc -u <user> > [...] > doveadm(<user>): Debug: Namespace inbox: type=private, prefix=INBOX., > sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:/home/.../test > doveadm(<user>): Debug: maildir++: root=/home/.../test, index=, > indexpvt=, control=, inbox=/home/.../test, alt= > doveadm(<user>): Debug: Namespace : type=private, prefix=, sep=, > inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none > doveadm(<user>): Debug: none: root=, index=, indexpvt=, control=, > inbox=, alt= > doveadm(<user>): Debug: quota: quota_over_flag check: quota_over_script > unset - skipping > doveadm(<user>): Debug: Quota root User quota: Recalculated relative > rules with bytes=268435456 count=0. Now grace=26843545 > doveadm(<user>): Debug: Namespace INBOX.: Using permissions from > /home/.../test: mode=0700 gid=default > > $ cat maildirsize > 268435456S > 29394 34 > > > So the used quota has exactly been doubled by just creating an empty > subfolder. > > Do you have any pointers for fixing my configuration or is this a bug in > dovecot? > > I coincidentally resurrected a months-old thread with this same issue a few days ago. I'm seeing the exact same after upgrading from 2.2.32 to 2.2.36. The original poster (who also narrowed it down to something in 2.2.34) mentioned a workaround that does indeed work, namely setting mailbox_list_index=no: > doveadm -o 'mailbox_list_index=no' quota recalc -u myuser I've been staring at diffs of 2.2.33 and 2.2.34 without anything jumping out at me (not a C guy, sadly). Maybe src/lib-storage/index/index-storage.c or src/lib-storage/list/mailbox-list-fs-iter.c or src/lib-storage/list/mailbox-list-index-iter.c or src/lib-storage/list/mailbox-list-index.c? The latter few have some added strcmp's against "INBOX". Then again, there's a lot of new code in the diffs under src/lib-storage that references INBOX specifically.

3 5

Dovecot replication and userdb "noreplicate".
by Reio Remma 07 Aug '19

07 Aug '19

Hello! I finally took the time and spent two days to set up replication for my server and now I have a question or two. I initially set noreplicate userdb field to 1 for all but a test user, but I could still see in the logs that all mailboxes were trying to connect to the other server via SSH. Is that normal? Jun 22 16:55:22 host dovecot: dsync-local(user(a)host.ee)<>: Error: Remote command returned error 84: ssh -i /home/vmail/.ssh/vmail.pem -l vmail backup.host.ee doveadm dsync-server -D -u user(a)host.ee Then I ended up setting mail_replica in userdb for only my test user, but I could still see in the logs that it was trying to sync the others as well, despite mail_replica being 0 for the rest. Jun 22 20:52:59 host dovecot: doveadm(user(a)host.ee): Fatal: -N parameter requires syncing with remote host I also notice (and read from recent posts) that sieve script replication doesn't work at all. Dovecot v2.3.6 and Pigeonhole from the official Dovecot CentOS repo. Thanks, Reio PS: Getting SSH for Dovecot to work with SELinux on CentOS 7 was fun as usual. :)

3 6

Incorrect link on download page
by Peter 06 Aug '19

06 Aug '19

The link to the latest 2.2 release points to 2.2.36.1 but the latest 2.2 is actually 2.2.36.3. Peter

2 1

doveadm: Error: open(/proc/self/io) failed
by me＠tdiehl.org 06 Aug '19

06 Aug '19

Hi, I am running dovecot-2.2.36-3.el7.x86_64 on a Centos 7 machine. I keep seeing the following errors in the dovecot.log: Jul 22 12:52:04 vmail2 dovecot: doveadm: Error: open(/proc/self/io) failed: Permission denied Dovecot -n is listed below: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24 (124e06aa) # OS: Linux 3.10.0-957.21.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: vmail2.kmg.mydomain.com auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN deliver_log_format = from=%{from}, envelope_sender=%{from_envelope}, subject=%{subject}, msgid=%m, size=%{size}, %$ dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } doveadm_password = # hidden, use -P to show it doveadm_port = 2525 first_valid_uid = 2000 last_valid_uid = 2000 mail_gid = 2000 mail_location = maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/ mail_plugins = quota mailbox_alias acl mail_log notify stats replication mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox "Junk E-mail" { auto = no special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%Ld/%%Ln prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl fts_autoindex = yes fts_autoindex_max_recent_msgs = 50 imapsieve_mailbox1_before = file:/usr/lib64/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/lib64/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge mailbox_delete mailbox_rename mail_log_fields = uid box msgid size from subject mail_replica = tcp:vmail1.kmg.mydomain.com mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent quota = dict:user::proxy::quotadict quota_grace = 10%% quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u sieve = ~/sieve/dovecot.sieve sieve_before = /var/vmail/sieve/dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /var/vmail/sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute sieve_max_redirects = 30 sieve_pipe_bin_dir = /usr/lib64/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_vacation_send_from_recipient = yes stats_refresh = 30 secs stats_track_cmds = yes } protocols = pop3 imap sieve lmtp service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service config { unix_listener config { user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 2525 } user = vmail } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { address = 127.0.0.1 port = 4190 } } service pop3-login { service_count = 1 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0666 } } service stats { fifo_listener stats-mail { mode = 0644 user = vmail } inet_listener { address = 127.0.0.1 port = 24242 } } ssl = required ssl_cert = </etc/pki/tls/certs/my_cert.crt ssl_cipher_list = ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5 ssl_client_ca_file = /etc/pki/tls/cert.pem ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes syslog_facility = local5 userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } protocol lda { lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_plugins = quota mailbox_alias acl mail_log notify stats replication sieve } protocol lmtp { lmtp_save_to_detail_mailbox = yes mail_plugins = quota sieve recipient_delimiter = + } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_max_userip_connections = 30 mail_plugins = quota mailbox_alias acl mail_log notify stats replication imap_quota imap_acl imap_stats imap_sieve } protocol pop3 { mail_max_userip_connections = 30 mail_plugins = quota mailbox_alias acl mail_log notify stats replication pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } Everything seems to be working properly and the dovecot wiki says that /proc/self/io is for statistics. I am thinking they are harmless but they are generating a bunch of noise in the logs and if harmless, I would like to silence them. Can someone let me know what causes this and if I should be concerned about this? My Google foo has not been helpful on this. Regards, -- Tom me(a)tdiehl.org

7 12

GSSAPI vs group check
by Németh Ákos Ferenc 05 Aug '19

05 Aug '19

Dear All, Is it possible to make any authorization (eg. checking of group membership) in case of GSSAPI authentication? Our dovecot authenticates the users against PAM and GSSAPI. In the PAM file I'm able to check if a user is a member of a selected (e.g mailreader) group. If the user is member, he can login otherwise not (see below). If the user has a valid Kerberos ticket and he tries to login via GSSAPI, I can't restrict him if he is not a member of the selected group. How can I overcome this issue? My config: passdb { driver = pam # [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=<n>] # [cache_key=<key>] [<service name>] #args = dovecot } userdb { # <doc/wiki/AuthDatabase.Passwd.txt> driver = passwd # [blocking=no] #args = # Override fields from passwd #override_fields = home=/home/virtual/%u } ...in PAM file: auth [success=1 default=ignore] pam_succeed_if.so user ingroup mailreader auth [success=ignore default=2] pam_succeed_if.so user ingroup admins auth [success=ignore default=1] pam_succeed_if.so uid >= 1000 auth [success=3 default=ignore] pam_winbind.so krb5_auth krb5_ccache_type=FILE cached_login auth [success=ignore default=1] pam_succeed_if.so uid < 1000 auth [success=1 default=ignore] pam_unix.so nullok_secure try_first_pass auth requisite pam_deny.so auth required pam_permit.so Thank you. Br, Ákos

2 3

doveadm sis deduplicate problem
by Peter Benko 03 Aug '19

03 Aug '19

Hi all, I'm running dovecot 2.3.7.1, with mdbox and single instance storage. I use offline deduplication like this: mail_attachment_fs = sis-queue /mailboxes/%d/_attachments_/queue:posix Each week I run doveadm sis deduplicate to deduplicate the mail attachments. Everything was fine until I mistakenly run doveadm sis deduplicate twice, simultaneously. I ended up having some error messages like this: Error: unlink(/mailboxes/domain.com/_attachments_/queue/067518ec61f872420813727326… failed: No such file or directory (in doveadm-sis.c:260) Also, the sis store has a suspicious temporary (leftover?) file (in the according _attachments_/06/75/ dir): -rw------- 4 vmail vmail 941197 Aug 2 15:48 temp.server-name.9633.77f5a7183d74f72b So the question is, shall I be nervous about this temp file? Does the error mean that the deduplication did not finish correctly? Is my mail store in a consistent state now? What shall I do with the temp file? I'd really appreciate some help... Regards, Peter

1 0

Sieve redirect
by B Fischer 03 Aug '19

03 Aug '19

What is the intended way to configure Sieve for the redirect command? Can I do it using in Dovecot (e.g. using the submission config) or does it have to be done in sendmail? I'm running Dovecot in a container and the MTA/MSA is in another container. Dovecot version: 2.3.2.1 (0719df592) - Björn

1 0

index failed
by ＠lbutlr 02 Aug '19

02 Aug '19

When reindexing I go the following error Mailbox test: UID=1: read(/path/to/mail,S=4248,W=4349) failed: Cached message size larger than expected (4248 > 1182, box=test, UID=1) (read reason=mail stream) In this case, the message was an unimportant test message with no content, so I simply removed it, but had it been important what would I need to do? -- Nine-tenths of the universe is the knowledge of the position and direction of everything in the other tenth. Every atom has its biography, every star its file, every chemical exchange its equivalent of the inspector with a clipboard. It is unaccounted for because it is doing the accounting for the rest of it. Nine-tenths of the universe, in fact, is the paperwork. --The Thief of Time

1 0

Question about replication
by Coy Hile 02 Aug '19

02 Aug '19

Good morning, I’m looking at dovecot replication for the first time for a particular use case: The VM running dovecot is currently in a public cloud, and I want to move it on-premises, so I want to replicate all mail data from the older instance to the new (then cutover internal DNS) (and in the process update to newer features on the internal instance such as mdbox vs Maildir). However, the new instance I’m building is internal, would have access to the internet only via NAT, so the replication would have to be one way. Is there anything in particular that I need to keep in mind when I’m trying to setup this sort of setup? The documentation that I’ve read seems to imply (or assume) synchronous, bidirectional replication. Thanks, and apologies if this is an obvious question; this is my first foray into doing something not utterly trivial with Dovecot.

2 3

Re: Panic: Module context expire_mail_user_module missing (Debian 10)
by Alexander Dalloz 02 Aug '19

02 Aug '19

Am 2019-08-02 11:05, schrieb Thomas Krause: Thomas, please keep the discussion on list. There is zero need to copy send any list discussion to my personal address. > # diff dovecot.conf dovecot.conf-ok > 5c5 > < mail_plugins = quota mailbox_alias acl zlib expire > --- >> mail_plugins = quota mailbox_alias acl zlib #expire > 170c170 > < expire = mysql:/etc/dovecot/dovecot-expire.conf > --- >> #expire = mysql:/etc/dovecot/dovecot-expire.conf > > > # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.4 () > doveconf: Warning: NOTE: You can get a new clean config file with: > doveconf -Pn > dovecot-new.conf > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:16: > ssl_dh_parameters_length is no longer needed > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:17: > ssl_protocols has been replaced by ssl_min_protocol > doveconf: Warning: NOTE: You can get a new clean config file with: > doveconf -Pn > dovecot-new.conf > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:16: > ssl_dh_parameters_length is no longer needed > doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:17: > ssl_protocols has been replaced by ssl_min_protocol Fix those obvious things. > # OS: Linux 4.19.0-5-amd64 x86_64 Debian 10.0 > # Hostname: mail.example.org > auth_mechanisms = PLAIN LOGIN > dict { > acl = mysql:/etc/dovecot/dovecot-share-folder.conf > expire = mysql:/etc/dovecot/dovecot-expire.conf > quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf > } > first_valid_uid = 1001 > last_valid_uid = 1001 > login_log_format_elements = user=<%u> rip=%r mpid=%e > session=<%{session}> > mail_gid = 1001 > mail_plugins = quota mailbox_alias acl zlib expire The expire plugin enabled under mail_plugins, but no configuration in the plugins { } section. So it has zeor function as you have not defined the mailboxes to track. > mail_uid = 1001 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart > extracttext > namespace { > inbox = yes > location = > mailbox Archive { > auto = no > special_use = \Archive > } > mailbox Archives { > auto = no > special_use = \Archive > } > mailbox "Deleted Messages" { > auto = no > special_use = \Trash > } > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox "Junk E-mail" { > auto = no > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Items" { > auto = no > special_use = \Sent > } > mailbox "Sent Messages" { > auto = no > special_use = \Sent > } > mailbox Spam { > auto = no > special_use = \Junk > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > separator = / > type = private > } > passdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > plugin { > acl = vfile > acl_shared_dict = proxy::acl > auth_socket_path = /var/run/dovecot/auth-master > mailbox_alias_new = Sent Messages > mailbox_alias_new2 = Sent Items > mailbox_alias_old = Sent > mailbox_alias_old2 = Sent > quota = dict:user::proxy::quotadict > quota_grace = 10%% > quota_rule = *:storage=1G > quota_warning = storage=100%% quota-warning 100 %u > quota_warning2 = storage=95%% quota-warning 95 %u > quota_warning3 = storage=90%% quota-warning 90 %u > quota_warning4 = storage=85%% quota-warning 85 %u > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > sieve_max_redirects = 30 > zlib_save = gz > zlib_save_level = 6 > } Above not expire configuration. See https://wiki.dovecot.org/Plugins/Expire And you should consider migrating to the autoexpunge settings. > protocols = pop3 imap sieve lmtp > service anvil { > client_limit = 1203 > } > service auth { > client_limit = 1400 > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-client { > group = www-data > mode = 0660 > } > unix_listener auth-master { > group = vmail > mode = 0666 > user = vmail > } > unix_listener auth-userdb { > group = vmail > mode = 0660 > user = vmail > } > } > service dict { > process_limit = 150 > unix_listener dict { > group = vmail > mode = 0660 > user = vmail > } > } > service imap-login { > process_limit = 500 > service_count = 1 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > user = vmail > } > service pop3-login { > process_limit = 500 > service_count = 1 > } > service quota-warning { > executable = script /usr/local/bin/dovecot-quota-warning.sh > unix_listener quota-warning { > group = vmail > mode = 0660 > user = vmail > } > } > ssl = required > ssl_cert = </etc/ssl/ssl.crt/mail.example.org.pem > ssl_cipher_list = > EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA > ssl_key = # hidden, use -P to show it > ssl_prefer_server_ciphers = yes > userdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > protocol lda { > auth_socket_path = /var/run/dovecot/auth-master > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > mail_plugins = quota mailbox_alias acl zlib expire sieve > postmaster_address = root > } > protocol lmtp { > lmtp_save_to_detail_mailbox = yes > mail_plugins = quota mailbox_alias acl zlib expire sieve > postmaster_address = postmaster > recipient_delimiter = + > } > protocol imap { > imap_client_workarounds = tb-extra-mailbox-sep > mail_max_userip_connections = 30 > mail_plugins = quota mailbox_alias acl zlib expire imap_quota > imap_acl > } > protocol pop3 { > mail_max_userip_connections = 30 > mail_plugins = quota mailbox_alias acl zlib expire > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > } Alexander

1 0

Panic: Module context expire_mail_user_module missing (Debian 10)
by Thomas Krause 02 Aug '19

02 Aug '19

Hi all, I tried to migrate a mailserver from Debian 9 (Dovecot 2.2.27) to Debian 10 (Dovecot 2.3.4.1). I moved the mail-partition and /etc/dovecot from the old to the new server. Dovecot started with 2 warnings. When trying to fetch mails via pop3 the server crashes: Aug 1 16:56:47 mail19 dovecot: pop3-login: Login: user=<testuser>, rip=::1, mpid=22394, session=<rutwcw+PRp4AAAAAAAAAAAAAAAAAAAAB> Aug 1 16:56:47 mail19 dovecot: pop3(testuser)<22394><rutwcw+PRp4AAAAAAAAAAAAAAAAAAAAB>: Panic: Module context expire_mail_user_module missing Aug 1 16:56:47 mail19 dovecot: pop3(testuser)<22394><rutwcw+PRp4AAAAAAAAAAAAAAAAAAAAB>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xdb13b) [0x7fd14ea7713b] -> /usr/lib/dovecot/libdovecot.so.0 (+0xdb1d1) [0x7fd14ea771d1] -> /usr/lib/dovecot/libdovecot.so.0(+0x4a001) [0x7fd14e9e6001] -> /usr/lib/dovecot/modules/lib20_expire_plugin.so(+0x23b2) [0x7fd14e7953b2] -> /usr/lib/dovecot/libdovecot-storage.so.0 (hook_mailbox_allocated+0x89) [0x7fd14eb89409] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_alloc+0xda) [0x7fd14eb84c5a] -> dovecot/pop3(client_init_mailbox+0x58) [0x55781fb5fba8] -> dovecot/pop3(+0x5645) [0x55781fb5e645] -> dovecot/pop3(+0x5882) [0x55781fb5e882] -> /usr/lib/dovecot/libdovecot.so.0(+0x6f601) [0x7fd14ea0b601] -> /usr/lib/dovecot/libdovecot.so.0(+0x6f97b) [0x7fd14ea0b97b] -> /usr/lib/dovecot/libdo vecot.so.0(+0x7030d) [0x7fd14ea0c30d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6f) [0x7fd14ea8d5ef] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136) [0x7fd14ea8ebe6] -> /usr/l ib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7fd14ea8d68c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fd14ea8d7f0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd14e a0dfd3] -> dovecot/pop3(main+0x2cf) [0x55781fb5e13f] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7fd14e7f909b] -> dovecot/pop3(_start+0x2a) [0x55781fb5e29a] Aug 1 16:56:47 mail19 dovecot: pop3(testuser)<22394><rutwcw+PRp4AAAAAAAAAAAAAAAAAAAAB>: Fatal: master: service(pop3): child 22394 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.h tml#coredumps) What can I do? I tried to enable core dumps (installed systemd-coredump and modified limits.conf) but it didn't work. Best regards, Thomas.

2 3

Patch needed for debian buster in version 2.2.36.1
by Simone Lazzaris 02 Aug '19

02 Aug '19

Hi all; I've just compiled version 2.2.36.1 on a new debian buster (debian 10) and it seems that this patch https://github.com/dovecot/core/commit/ 63a74b9e8e0604486a15a879e7f1a27257322400.patch is needed to make it work. Without the patch, the authentication segfaults. *Simone Lazzaris* *Qcom S.p.A.* simone.lazzaris(a)qcom.it[1] | www.qcom.it[2] * LinkedIn[3]* | *Facebook*[4] [5] -------- [1] mailto:simone.lazzaris@qcom.it [2] https://www.qcom.it [3] https://www.linkedin.com/company/qcom-spa [4] http://www.facebook.com/qcomspa [5] https://www.qcom.it/includes/NUOVAemail-banner.gif

2 1

IMAP frontend authenticating proxy with GSSAPI/Kerberos SSO
by Gert van Dijk 01 Aug '19

01 Aug '19

Hi, My IMAP backend server is lacking SSO authentication, so I am trying to set up Dovecot in front of it as an authenticating proxy. Fortunately, my backend server provides a way to ignore the password provided and will simply trust the username given to be authenticated, using plain login authentication. I'm struggling with setting this up, as it seems to me that as soon as I enable GSSAPI/Kerberos SSO in Dovecot it ignores my passdb proxy settings and it creates a local mail store instead. Is this behavior intentional, and why? See also my Q on Serverfault.com [1], where some user has reported this set up to be working with the same backend server and provided some hints, but I believe I'm hitting a combination that cannot be working with this version. Dovecot version: v2.3.4.1 (from Debian Buster) dovecot -n (I'm using only a single flat file dovecot.conf): ===== # dovecot -n # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.15.0-46-generic x86_64 Debian 10.0 overlay # Hostname: imap.sub.mydomain.tld auth_debug = yes auth_default_realm = MY-REALM.DOMAIN.TLD auth_gssapi_hostname = imap.sub.mydomain.tld auth_krb5_keytab = /etc/dovecot/imap.keytab auth_mechanisms = gssapi auth_realms = MY-REALM.DOMAIN.TLD auth_username_format = %n auth_verbose = yes log_path = /dev/stderr mail_gid = 65534 mail_location = mbox:/tmp mail_uid = 65534 passdb { args = proxy=y host=127.0.0.1 port=1143 pass=#hidden_use-P_to_show# driver = static } protocols = imap ssl = required ssl_cert = </etc/dovecot-ssl/cert.crt ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes ===== My comments are lost with "dovecot -n", so here I repeat some why's. ===== # uid/gid of nobody or else: # imap(username)<15><6SCG0QOPgtcKAAYE>: Error: Couldn't drop privileges: User is missing UID (see mail_uid setting) mail_uid = 65534 mail_gid = 65534 # Have to set dummy mail_location or else: # Error: mail_location not set and autodetection failed: Mail storage autodetection failed with home=(not set) mail_location = mbox:/tmp ===== (Ideally, I don't want to set any local storage storage, Dovecot should just be a proxy in my case.) Log, debug enabled, with client connecting: ===== master: Info: Dovecot v2.3.4.1 (f79e8e7e4) starting up for imap auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/lib20_auth_var_expand_crypt.so auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libmech_gssapi.so auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat auth: Debug: auth client connected (pid=9) auth: Debug: client in: AUTH 1 GSSAPI service=imap secured=tls session=9WOjSwWP8toKAAYE lip=9.9.9.9 rip=1.2.3.4 lport=143 rport=56050 local_name=sub.mydomain.tld ssl_cipher=ECDHE-RSA-AES128-GCM-SHA256 ssl_cipher_bits=128 ssl_pfs=KxECDHE ssl_protocol=TLSv1.2 auth: Debug: gssapi(?,1.2.3.4,<9WOjSwWP8toKAAYE>): Obtaining credentials for imap(a)sub.mydomain.tld auth: Debug: client passdb out: CONT 1 auth: Debug: client in: CONT<hidden> auth: Debug: gssapi(username,1.2.3.4,<9WOjSwWP8toKAAYE>): security context state completed. auth: Debug: client passdb out: CONT 1 YIGZBgkqhki[...]7Bm1G auth: Debug: client in: CONT<hidden> auth: Debug: gssapi(username,1.2.3.4,<9WOjSwWP8toKAAYE>): Negotiated security layer auth: Debug: client passdb out: CONT 1 BQQF/w[...]yjNOo= auth: Debug: client in: CONT<hidden> auth: Debug: static(username,1.2.3.4,<9WOjSwWP8toKAAYE>): lookup auth: Info: static(username,1.2.3.4,<9WOjSwWP8toKAAYE>): No password returned (and no nopassword) auth: Debug: client passdb out: OK 1 user=username original_user=username(a)MY-REALM.DOMAIN.TLD auth: Debug: master in: REQUEST 3940810753 9 1 740b6a075ea21dbe10397f3734262bbe session_pid=12 request_auth_token auth: Debug: master userdb out: USER 3940810753 username auth_token=d1503de7036a68a85ea9db25f32bc9df3ebf08a8 auth_user=username(a)MY-REALM.DOMAIN.TLD imap-login: Info: Login: user=<username>, method=GSSAPI, rip=1.2.3.4, lip=9.9.9.9, mpid=12, TLS, session=<9WOjSwWP8toKAAYE> ===== The good news is that everything related to Kerberos/GSSAPI authentication seems to work fine! Noteworthy in the above output I think is the line "auth: Debug: client passdb out: ..." which does not mention any proxy related settings any longer which aligns with my observations: - The client is then logged into an empty mailbox in local storage. Not okay. - No attempt to connect to my backend server is made (checked with tcpdump). Not okay. Somehow I start to believe that as soon as I set auth_mechanisms = gssapi, any args/field in passdb are ignored. Is that a bug or am I looking at a feature? What else have I tried: - Using imapc proxy [2], but that example config does not work in my Dovecot version with the error: auth: Error: prefetch(username,1.2.3.4,): userdb lookup not possible with only userdb prefetch - Using a passdb with master=yes and pass=yes. passdb { driver = static args = proxy=y host=127.0.0.1 port=1143 pass=dummypass master = yes pass = yes } This fails with the error: auth: Fatal: Master passdb can't have pass=yes if there are no passdbs master: Error: service(auth): command startup failed, throttling for 2 secs imap-login: Info: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=1.2.3.4, lip=9.9.9.9, session=<0i46qgWPntsKAAYE> imap-login: Warning: Timeout leak: 0x7efe483e8120 (auth-server-connection.c:397) Any help or clues are very much appreciated. Thanks! [1]: https://serverfault.com/q/951010/135437 [2]: https://wiki.dovecot.org/HowTo/ImapcProxy Gert van Dijk

2 2

qoutas for local domains
by Benny Pedersen 01 Aug '19

01 Aug '19

will it be possible to let local users have free qoutas while remote domains is qouta limited ? my wish is to let 2 local domains be qouta free, if possible, if not i like to let it be work to do, if i can provide a patch

1 0

dovecot/imap crashes on Centos 7
by marvin alcantara 30 Jul '19

30 Jul '19

Hi All, I have my dovecot/imap crashing on my Centos7 server. Below is the stacktrace I got: #0 0x00007f32d9b84e58 in array_count_i (array=0x55b332a1f8f8) at array.h:155 No locals. #1 array_get_i (count_r=<synthetic pointer>, array=0x55b332a1f8f8) at array.h:209 No locals. #2 seq_range_lookup (array=0x55b332a1f8f8, seq=7715, idx_r=0x7ffe5d719574) at seq-range-array.c:14 data = <optimized out> idx = <optimized out> left_idx = <optimized out> right_idx = <optimized out> count = <optimized out> #3 0x00007f32d9b859ae in seq_range_exists (array=array@entry=0x55b332a1f8f8, seq=seq@entry=7715) at seq-range-array.c:391 idx = 32766 #4 0x00007f32d9e4b3d9 in mailbox_search_result_add (result=result@entry=0x55b332a1f8e0, uid=7715) at mailbox-search-result.c:121 __FUNCTION__ = "mailbox_search_result_add" #5 0x00007f32d9eb0f30 in index_search_result_update_appends (result=result@entry=0x55b332a1f8e0, old_messages_count=<optimized out>) at index-search-result.c:168 t = 0x55b332a417d0 search_ctx = 0x55b332a1a580 mail = 0x55b332ae8e58 search_arg = {next = 0x55b332a7c0a0, type = SEARCH_SEQSET, value = {subargs = 0x0, seqset = { arr = {buffer = 0x55b3329da360, element_size = 8}, v = 0x55b3329da360, v_modifiable = 0x55b3329da360}, str = 0x0, time = 0, size = 0, flags = 0, search_flags = (unknown: 0), date_type = 0, thread_type = MAIL_THREAD_NONE, modseq = 0x0, search_result = 0x0, mime_part = 0x0}, initialized = {search_args = 0x0, keywords = 0x0, mailbox_glob = 0x0}, context = 0x0, hdr_field_name = 0x0, match_not = 0, match_always = 0, nonmatch_always = 0, fuzzy = 0, no_fts = 0, result = -1} message_count = 160 ret = <optimized out> __FUNCTION__ = "index_search_result_update_appends" #6 0x00007f32d9ebbc37 in search_result_update (result=0x55b332a1f8e0, ctx=0x55b332a2ed90) at index-sync-search.c:78 No locals. #7 index_sync_search_results_update (ctx=ctx@entry=0x55b332a2ed90) at index-sync-search.c:88 results = 0x55b332a3b4a0 i = <optimized out> count = <optimized out> #8 0x00007f32d9ebc4b4 in index_mailbox_sync_deinit (_ctx=0x55b332a2ed90, status_r=0x55b332a1f858) at index-sync.c:353 ctx = 0x55b332a2ed90 sync_rec = {seq1 = 0, seq2 = 0, type = MAILBOX_SYNC_TYPE_EXPUNGE} delayed_expunges = true ret = <optimized out> #9 0x00007f32d9e9e400 in mailbox_list_index_sync_deinit (ctx=0x55b332a2ed90, status_r=0x55b332a1f858) at mailbox-list-index.c:976 ibox = 0x55b332a3f990 box = 0x55b332a3b098 #10 0x00007f32d9e3b568 in mailbox_sync_deinit (_ctx=<optimized out>, status_r=<optimized out>) at mail-storage.c:1953 ctx = <optimized out> box = <optimized out> errormsg = <optimized out> error = MAIL_ERROR_NONE ret = <optimized out> #11 0x000055b33188c431 in imap_sync_finish (ctx=ctx@entry=0x55b332a1f7e0, aborting=aborting@entry=false) at imap-sync.c:279 client = 0x55b3329fb6a8 ret = 0 #12 0x000055b33188cfed in imap_sync_more (ctx=0x55b332a1f7e0) at imap-sync.c:594 str = <optimized out> ret = 1 __FUNCTION__ = "imap_sync_more" #13 0x000055b33188d676 in cmd_sync_continue (sync_cmd=0x55b3329fce58) at imap-sync.c:629 cmd = 0x55b3329f5b20 prev = <optimized out> client = 0x55b3329fb6a8 ctx = 0x55b332a1f7e0 ret = <optimized out> __FUNCTION__ = "cmd_sync_continue" #14 0x000055b33188db34 in cmd_sync_client (sync_cmd=0x55b3329fce58) at imap-sync.c:720 ctx = <optimized out> flags = <optimized out> client = 0x55b3329fb6a8 imap_flags = <optimized out> no_newmail = <optimized out> #15 cmd_sync_delayed_real (client=0x55b3329fb6a8) at imap-sync.c:833 cmd = <optimized out> first_expunge = <optimized out> first_nonexpunge = <optimized out> #16 cmd_sync_delayed (client=client@entry=0x55b3329fb6a8) at imap-sync.c:841 _data_stack_cur_id = 3 #17 0x000055b331873707 in client_input_append (cmd=0x55b3329fce58) at cmd-append.c:121 ctx = 0x55b3329fcff0 client = 0x55b3329fb6a8 reason = <optimized out> finished = <optimized out> lit_offset = <optimized out> __FUNCTION__ = "client_input_append" #18 0x00007f32d9b72be2 in io_loop_call_io (io=0x55b3329f5c70) at ioloop.c:600 ioloop = 0x55b3329e29c0 t_id = 2 __FUNCTION__ = "io_loop_call_io" #19 0x00007f32d9b742cf in io_loop_handler_run_internal (ioloop=ioloop@entry=0x55b3329e29c0) at ioloop-epoll.c:223 ctx = 0x55b3329e44e0 events = <optimized out> list = 0x55b3329e5f40 io = <optimized out> tv = {tv_sec = 3, tv_usec = 487057} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 call = <optimized out> __FUNCTION__ = "io_loop_handler_run_internal" #20 0x00007f32d9b72c7c in io_loop_handler_run (ioloop=ioloop@entry=0x55b3329e29c0) at ioloop.c:649 No locals. #21 0x00007f32d9b72e38 in io_loop_run (ioloop=0x55b3329e29c0) at ioloop.c:624 __FUNCTION__ = "io_loop_run" #22 0x00007f32d9af7c03 in master_service_run (service=0x55b3329e2860, callback=callback@entry=0x55b33188de30 <client_connected>) at master-service.c:724 No locals. #23 0x000055b331873262 in main (argc=1, argv=0x55b3329e25f0) at main.c:484 set_roots = {0x55b331a9e700 <imap_setting_parser_info>, 0x7f32da3475a0, 0x0} login_set = {auth_socket_path = 0x55b3329da048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x55b33188e600 <login_client_connected>, failure_callback = 0x55b33188df40 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 auth_socket_path = <optimized out> c = <optimized out> Dovecot version: dovecot-2.2.36-3.el7.x86_64 I would like to ask for your inputs regarding this.Thank you very much for your help! Regards,Marvin

1 0

Bug with latest GCC 9
by Martin Liška 29 Jul '19

29 Jul '19

Hi. As mentioned here: https://bugzilla.opensuse.org/show_bug.cgi?id=1123136 there's a new issue with GCC 9, it's related to: https://gcc.gnu.org/gcc-9/porting_to.html#complit in: /* Not const! Never return this as a result directly! */ #define SMTP_ADDRESS_LITERAL(localpart, domain) \ &((struct smtp_address){ (localpart), (domain) }) Thanks, Martin

2 4

sieve scripts not synching for 2.3.5.1 pre-built
by Jan-Pieter Cornet 29 Jul '19

29 Jul '19

Hi, We're synching mailboxes, changing format from maildir to mdbox, using doveadm backup/doveadm sync. When still running 2.2.36, 'doveadm backup' also synched the sieve scripts, without issues. After the upgrade to 2.3.5.1, the sieve sync stopped working. We're using the pre-built 2.3 packages from https://repo.dovecot.org/ce-2.3-latest/debian/stretch In fact, when I now remove the dovecot-2.3 packages, and use my own old 2.2.36 packages, using the exact same config, the sieve scripts are being copied again. In 2.2.36, when I migrate a mailbox, I do see Pigeonhole being initialized. Eg: userimap-dev1:~ # doveadm -D backup -u xtra30 "doveadm -D dsync-server -u xtra30+mdbox=.8d1&4&03&xtra30" (note: that strange destination user signals to the auth proxy that we want the new mdbox storage instead of the default for this user) Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so [...skipping a bunch of lines...] dsync-local(xtra30): Debug: brain M: Mailbox Spam: local=88b39d12998d9e5cbb700000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30): Debug: brain M: Mailbox Test: local=801bb932e98d9e5c3d6d0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30): Debug: brain M: Mailbox Trash: local=fc9b162b86479f5cc85a0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-remote(xtra30): Debug: brain S: Skipping unchanged mailbox e7c1961d84479f5c885a0000436edcce dsync-local(xtra30): Debug: doveadm-sieve: Iterating Sieve mailbox attributes dsync-local(xtra30): Debug: sieve: Pigeonhole version 0.4.24 () initializing dsync-local(xtra30): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. dsync-local(xtra30): Debug: sieve: file storage: Using active Sieve script path: /var/mail/.8d1/index/7/oldsieve/x/xt/xtra30/dovecot.sieve [... skipping a lot of sieve debug lines that seem fine ...] dsync-remote(xtra30): Debug: sieve: file script: Opened script `xtra30-testfilter' from `/var/mail/.8d1/index/4/03/xtra30/sieve/scripts/xtra30-testfilter.sieve' dsync-remote(xtra30): Debug: doveadm-sieve: Assigned value for key `vendor/vendor.dovecot/pvt/server/sieve/default' (last change: 2019-03-29 22:27:41) dsync-remote(xtra30): Debug: brain S: Import INBOX: Import attribute vendor/vendor.dovecot/pvt/server/sieve/default: Nonexistent locally dsync-remote(xtra30): Debug: brain S: Import INBOX: Last common UID=0. Delayed expunges= dsync-remote(xtra30): Debug: brain S: Import INBOX: Import change type=save GUID=1553894809.M254847P28859.userimap11.xs4all.net,S=1311,W=1342 UID=1 hdr_hash= result=Mail's UID is above local UIDNEXT - No more local mails found [...] Afterwards, the sieve scripts have been transferred, even including last-modified time of any scripts. However, with 2.3.5.1, it seems doveadm-sieve is never called from dsync. Output includes: userimap-dev1:~ # doveadm -D backup -u xtra30 "doveadm -D dsync-server -u xtra30+mdbox=.8d1&4&03&xtra30" Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so [... skipping some lines again ..] dsync-local(xtra30)<SCSfIc5Wo1zTKAAAsPW7qw>: Debug: brain M: Mailbox Spam: local=88b39d12998d9e5cbb700000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30)<SCSfIc5Wo1zTKAAAsPW7qw>: Debug: brain M: Mailbox Test: local=801bb932e98d9e5c3d6d0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30)<SCSfIc5Wo1zTKAAAsPW7qw>: Debug: brain M: Mailbox Trash: local=fc9b162b86479f5cc85a0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-remote(xtra30)<h8hfIs5Wo1zWKAAAsPW7qw>: Debug: brain S: Skipping unchanged mailbox e7c1961d84479f5c885a0000436edcce dsync-remote(xtra30)<h8hfIs5Wo1zWKAAAsPW7qw>: Debug: brain S: Import INBOX: Last common UID=0. Delayed expunges= dsync-remote(xtra30)<h8hfIs5Wo1zWKAAAsPW7qw>: Debug: brain S: Import INBOX: Import change type=save GUID=1553894809.M254847P28859.userimap11.xs4all.net,S=1311,W=1342 UID=1 hdr_hash= result=Mail's UID is above local UIDNEXT - No more local mails found [...] In the above output from 2.3.5.1, all lines starting with "Debug: doveadm-sieve: Iterating Sieve mailbox attributes" are absent. I have no idea why. We can "work around" it by explicitly doing an rsync of the sieve storage after the migration, but that's a bit of a hack. Attached is a doveconf -n output for 2.3.5.1. -- Jan-Pieter Cornet <johnpc(a)xs4all.net> Systeembeheer XS4ALL Internet bv www.xs4all.nl

6 7

IMAP copy stopped copying flags
by Andrzej A. Filip 28 Jul '19

28 Jul '19

Is it intended behavior? It seems to be caused by upgrade to 1:2.3.2-2 on Debian/Testing. -- A. Filip

4 6

submission configuration issues
by Jean-Daniel Dupas 28 Jul '19

28 Jul '19

Hello, I'm having trouble configuring the submission proxy. I have configured the submission service as follow: submission_host = smtp.example.com submission_relay_host = localhost submission_relay_port = 8587 submission_relay_rawlog_dir = /var/log/dovecot/ submission_relay_trusted = yes My main issue is that until I login, dovecot-submission won't connect to the backend and query the capabilities and so won't report the right capabilities. That mean that the first EHLO message don't get the right capabilities list. " EHLO example.com 250-smtp.example.com 250-8BITMIME 250-AUTH PLAIN LOGIN 250-BURL imap 250-CHUNKING 250-ENHANCEDSTATUSCODES 250-SIZE 250 PIPELINING " This list don't contains VRFY, DNS, and SIZE is not specified (all of these is present in backend EHLO response). After login, if I send an new EHLO command, everything is properly reported. The raw log shows that unlike what the documentation says, dovecot don't try to connect to the backend until the user is properly logged. In my raw log I show that after I logged in dovecot-submission, the later open a connection to the backend and send a X-CLIENT command. Now, if I try to force the capabilities by using: submission_backend_capabilities = VRFY 8BITMIME DSN dovecot properly reports all SMTP capabilities in the first EHLO response, but it completely stops emitting X-CLIENT command to the backend and try to simply forward the command without authentication, which result in postfix rejecting the command with an unauthorized user error. What is wrong with my configuration ? Thanks. Jean-Daniel

4 7

Help with IMAP IDLE
by Jorge Bastos 28 Jul '19

28 Jul '19

Hi, I'm becaming crazy about this! I've asked before, not no matter what i do or conf, never works. How can i make IDLE work for real with MSOutlook, so that i can receive emails when they arrive, instead of changing folder/get out and in my account? Here's my conf, should it be reflected in postconf -n (is it the running conf?): ================== root@fastmail:/etc/dovecot# grep -ir idle * conf.d/20-imap.conf.ucf-dist:# If nothing happens for this long while client is IDLEing, move the connection conf.d/20-imap.conf.ucf-dist:# IDLEing. conf.d/20-imap.conf.ucf-dist:#imap_idle_notify_interval = 2 mins conf.d/20-imap.conf:imapc_max_idle_time = 1440 mins conf.d/20-imap.conf:# IDLEing. conf.d/20-imap.conf:#imap_idle_notify_interval = 2 mins conf.d/20-imap.conf:imap_idle_notify_interval = 1 min conf.d/20-imap.conf:#imap_client_workarounds = outlook-idle conf.d/10-mail.conf.ucf-dist:# When IDLE command is running, mailbox is checked once in a while to see if conf.d/10-mail.conf.ucf-dist:#mailbox_idle_check_interval = 30 secs conf.d/10-mail.conf:# When IDLE command is running, mailbox is checked once in a while to see if conf.d/10-mail.conf:#mailbox_idle_check_interval = 30 secs root@fastmail:/etc/dovecot# postconf -n|grep -i idle root@fastmail:/etc/dovecot#

7 16

Since multiple ,matches
by ＠lbutlr 27 Jul '19

27 Jul '19

I have a rule in sieve like the following: if anyof (header :contains ["to"] [ "box", "change", "cornell”, “twitter”]) { setflag "\\Seen"; fileinto :create "misc"; stop; } The messages that come in are using address extensions, so user+twitter(a)example.com, for example. The trouble is, this sieve recipe doesn’t get triggered and the message gets filed into the extension folder “twitter” or “cornell” instead of “misc”. I have a very similar recipe following that one: if header :contains "to" "root" { setflag "\\Seen"; addflag "$label5"; fileinto :create "root"; stop; } And that one works as I expect. Is that multiple match format valid for :contains (I’ve only seen it used for :regex)? I don’t get an error. Is there a way to run a debug log for the sieve recipe? Preferably for a single user. -- Vernon: Now this is the thought that wakes me up in the middle of the night. That when I get older, these kids are going to take care of me Carl: I wouldn't count on it.

1 1

Re: Dovecot, FreeBSD, and Solr?
by J. de Meijer 27 Jul '19

27 Jul '19

> Hello, > > Does anyone have this combination going? I'm running freebsd and dovecot, to which i'd like to add solr text search. I am stuck on xml schema integration, if anyone has any notes i'd appreciate them. I basically followed https://www.c0ffee.net/blog/mail-server-guide/ And dovecot with solr is running fine on freebsd for me.

3 3

corrupt mdbox index / zero mails showing in imap
by Mike 27 Jul '19

27 Jul '19

Hi, I have recently migrated (under emergency conditions) a dovecot imap/pop based server to a new instance. The mailboxes used mdbox format and due to various screwups I had corrupt indexes. I thought I'd cleaned this up but then I found that this new instance hadn't been set up correctly for nfs. Long story short, I still get users with new cases of corrupt indexes. The symptom is imap either showing NO mail in their inbox, or, not showing any recently delivered mail in the box, until I rm -f dovecot.map.index / doveadm force-resync -u user. It would be a huge help if there could be some method to detect if this is the case for any given user and to proactively do the force-resync process for them instead of waiting for that support call (or service cancellation...). I have looked around and have not found any tool capable of 'linting' an mdbox format inbox, and it seems like something like this should have been or would be an extremely useful debugging tool both during development as well as to troubleshoot stuff in the field. I would love to know if anyone either has such a tool, or any general suggestion how I could go about finding these cases and addressing them. I believe I have the nfs issue resolved and will not be creating new cases, so I just want to fix the ~3000 boxes I have now and move forward. Thank you. Mike-

2 2

doveadm migration to server with no root access
by jon rose 26 Jul '19

26 Jul '19

Hi I am trying to do mailbox migrations using doveadm to a server that I do not have root access on without success. Is this possible? What I have done and results: ssh to server with mailboxes to copy and then su cp /etc/dovecot/dovecot.conf /root/tmp/dovecot_new.conf append text below** to end of dovecot_new.conf try to copy mail 3 different ways (all fail see error messages) doveadm -D -v -c /root/tmp/dovecot_new.conf -o mail_fsync=never sync -1 -u test(a)propertyzone.co.za pop3c: fail with: Error: Mailbox INBOX.Archive sync: mailbox_set_subscribed failed: Subscriptions not supported doveadm -D -v -c /root/tmp/dovecot_new.conf -o mail_fsync=never sync -u test(a)propertyzone.co.za pop3c: fail with: Error: Can't create mailbox INBOX.Archive: POP3 mailbox creation isn't supported doveadm -D -v -c /root/tmp/dovecot_new.conf -o mail_fsync=never backup -u test(a)propertyzone.co.za pop3c: fail with: dsync(test(a)propertyzone.co.za): Panic: file mailbox-list-delete.c: line 473 (mailbox_list_delete_symlink_default): assertion failed: (ret > 0) ** append at end of dovecot_new.conf pop3c_host = dedi225.jnb3.host-h.net pop3c_user = test(a)propertyzone.co.za pop3c_password = ***** namespace inbox { inbox = yes } namespace { prefix = POP3-MIGRATION-NS/ location = pop3c: list = no hidden = yes } protocol doveadm { mail_plugins = $mail_plugins pop3_migration } plugin { pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX -- Sent from: http://dovecot.2317879.n4.nabble.com/

2 4

Dovecot, FreeBSD, and Solr?
by David Mehler 26 Jul '19

26 Jul '19

Hello, Does anyone have this combination going? I'm running freebsd and dovecot, to which i'd like to add solr text search. I am stuck on xml schema integration, if anyone has any notes i'd appreciate them. Thanks. Dave.

2 1

dsync crash
by tono 25 Jul '19

25 Jul '19

Hello, I am getting a persistent seg fault while trying to sync our server to a new location. We are on 2.2.33.2, due for an upgrade but we have used dsync successfully in the past. In fact, we used it to populate this same instance. So I hope this is just a configuration issue. My logs are being filled with these errors:: 2019-07-10T08:29:53-04:00 localhost dovecot: [ID 583609 mail.crit] doveadm: Fatal: master: service(doveadm): child 44707 killed with signal 11 (core dumped) On the remote we see: 2019-07-10T10:17:24-04:00 localhost dovecot: [ID 583609 mail.error] doveadm: Error: read(111.111.111.111) failed: EOF (version not received) I tried doing sync manually: truss -rall -wall -f doveadm sync -u tpyro(a)metro-email.com tcp:123.123.123.123 55440: V E R S I O N\t d o v e a d m - s e r v e r\t 1\t 1\n 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: read(10, " -\n", 8192) = 2 55440: write(10, 0x08126598, 31) = 31 55440: P L A I N\t P A S s w o R d R E D A C t E D x W o e N X H h\n 55440: read(10, 0x0814980A, 8190) Err#11 EAGAIN 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: read(10, 0x0814980A, 8190) = 27 55440: V E R S I O N\t d o v e a d m - c l i e n t\t 1\t 1\n 55440: read(10, 0x08149825, 8163) Err#11 EAGAIN 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: read(10, " +\n", 8163) = 2 55440: write(10, 0x081738D0, 60) = 60 55440: \t u s e r 1 @ d o m a i n - n a m e . c o m\t d s y n c - s e r 55440: v e r\t - u u s e r 1 @ d o m a i n - n a m e . c o m\n 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: Incurred fault #6, FLTBOUNDS %pc = 0xFEC84490 55440: siginfo: SIGSEGV SEGV_MAPERR addr=0xD89CDB4E 55440: Received signal #11, SIGSEGV [default] 55440: siginfo: SIGSEGV SEGV_MAPERR addr=0xD89CDB4E I saw the note in the wiki about "dovecot --hostdomain" so I made sure they do not match. The new target (z-imap2) was built using a zfs clone from the source (z-imap1). I changed the 'hostname', 'instance_name", and 'mail_replica' fields but otherwise they are identical. Did I miss something? Thanks, Tony # 2.2.33.2 (d6601f4ec): /opt/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: SunOS 5.11 i86pc auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_default_realm = domain-name.com auth_mechanisms = plain login auth_verbose = yes default_process_limit = 500 default_vsz_limit = 512 M doveadm_password = # hidden, use -P to show it doveadm_port = 10993 hostname = z-imap1 instance_name = dovecot-2.2.13 # Note this was a reference to an earlier install, just an oversight. True version is 2.2.33.2. mail_location = mdbox:%h/mdbox mail_plugins = " mail_log notify replication" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox "Deleted Messages" { special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = } prefix = separator = / } passdb { args = scheme=CRYPT username_format=%Lu /opt/local/etc/dovecot/users driver = passwd-file } plugin { imapsieve_mailbox1_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-spam.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_name = Spam imapsieve_mailbox3_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-ham.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_from = Junk imapsieve_mailbox3_name = INBOX imapsieve_mailbox4_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-ham.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Spam imapsieve_mailbox4_name = INBOX imapsieve_mailbox5_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-ham.sieve imapsieve_mailbox5_causes = COPY imapsieve_mailbox5_from = * imapsieve_mailbox5_name = NotJunk mail_replica = tcp:123.123.123.123 # True IP redacted sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /opt/local/etc/dovecot/sieve/spam-training/spam-to-folder.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve/spam-training sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_dir = /opt/local/etc/dovecot/sieve/trace sieve_trace_level = matching } postmaster_address = postmaster(a)domain-name.com protocols = imap lmtp sieve replication_max_conns = 1 service aggregator { fifo_listener replication-notify-fifo { user = doveuser } unix_listener replication-notify { user = doveadm } } service anvil { client_limit = 1503 } service auth { client_limit = 2500 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0660 } } service doveadm { inet_listener { port = 10993 } vsz_limit = 4 G } service imap { executable = imap postlogin vsz_limit = 3 G } service indexer-worker { vsz_limit = 4 G } service lmtp { inet_listener lmtp { port = 51924 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service postlogin { executable = script-login rawlog -t } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl_cert = </opt/custom/etc/ssl/wildcard.domain-name.com/wildcard.domain-name.com.bundled.crt ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /opt/local/etc/dovecot/users driver = passwd-file } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 24 mail_plugins = mail_log notify replication imap_sieve } protocol lmtp { mail_plugins = mail_log notify replication sieve }

1 1

lmtp with non ASCII Parameters changed in dovecot 2.3
by Matthias Lay 25 Jul '19

25 Jul '19

Hi, I updated from 2.2 to 2.3 and noticed lmtp rejects parameters containing non ASCII now. this worked with dovecot-2.2 lmtp. the string is already denied after reading in "smtp_command_parse_parameters()" mail from:<bluö> 500 5.5.2 Invalid command syntax is there a plan to change this behaviour, or to implement the SMTPUTF8 extension in lib-smtp? I guess thie would fix Michals "Dovecot LMTP rejecting mail from address with apostrophe" Problem too. Greetz Matze

1 1

Sieve adding header content to a different header
by ＠lbutlr 24 Jul '19

24 Jul '19

Given a message itht he headers: Subject: This is the subject: X-Foo: bar how would I do the following: 1) Add the contents of X-Foo to subject Subject: [bar] This is the subject 2) replicate the X-Foo header into X-temp X-temp: X-foo-bar -- THE PRESIDENT DID IT IS NOT AN EXCUSE Bart chalkboard Ep. AABF05

2 2

Freebsd ports?
by ＠lbutlr 23 Jul '19

23 Jul '19

The version of dovecot in freebsd ports tree is still 2.3.6, any idea when this will be updated to 2.3.7? -- "I can't see the point in the theatre. All that sex and violence. I get enough of that at home. Apart from the sex, of course." - Baldrick

2 2

Dovecot v2.3.7.1 & Pigeonhole v0.5.7.1 released
by Timo Sirainen 23 Jul '19

23 Jul '19

https://dovecot.org/releases/2.3/dovecot-2.3.7.1.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.7.1.tar.gz.sig https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.1.… https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.1.… Binary packages in https://repo.dovecot.org/ These releases fix the reported regressions in v2.3.7 & v0.5.7. Dovecot core: - Fix TCP_NODELAY errors being logged on non-Linux OSes - lmtp proxy: Fix assert-crash when client uses BODY=8BITMIME - Remove wrongly added checks in namespace prefix checking Pigeonhole: - dsync: Sieve script syncing failed if mailbox attributes weren't enabled.

2 1

Fwd: Character not allowed in mailbox name
by Lothar Schilling 23 Jul '19

23 Jul '19

Hi Arnold, thanks for your assistance. I solved the issue right now by changing prefix = Shared/%%u/ to prefix = Shared/%%n/ This omits the @mydomain.de part which holds the dot causing the trouble. Why this became a problem due to the update I wouldn't know. Regards Lothar Am 22.07.2019 um 10:49 schrieb Arnold Opio Oree: > Hi Lothar, > > I haven't yet had the opportunity to go deep into Dovecot engineering / > development, (so please correct me if necessary anyone). > > I have experienced this exception where attempting to copy mailbox data > within a client from IMAP account to IMAP account, but I have not had > it cause a mailbox to become inaccessible. > > The reason was because Maildir++ format uses the "." separator to > delineate child directories, and therefore the character is illegal in > mailbox names. The issue was addressed by either changing the > filesystem layout to LAYOUT=fs or by changing the mailbox format > altogether to e.g. sdbox. > >> location = maildir:%%h:INDEX=%h/shared/%%u > I can see from your configuration that the LAYOUT is not set as fs, so > I would assume is still using Maildir++. > > Perhaps this setting was lost during the upgrade? > > Regards, > > Arnold Opio Oree > Chief Executive Officer > Parallax Digital Technologies > > arnoldoree(a)parallaxdt.com > > http://www.parallaxdt.com > > tel : +44 (0) 333 577 8587 > fax : +44 (0) 20 8711 2477 > > Parallax Digital Technologies is a trading name of Parallax Global > Limited. U.K. Co. No. 08836288 > > The contents of this e-mail are confidential. If you are not the > intended recipient you are to delete this e-mail immediately, disregard > its contents and disclose them to no other persons. > > -----Original Message----- > From: Lothar Schilling via dovecot <dovecot(a)dovecot.org> > Reply-To: Lothar Schilling <ls(a)proasyl.de> > To: dovecot(a)dovecot.org > Subject: Character not allowed in mailbox name > Date: Mon, 22 Jul 2019 08:33:45 +0200 > > Hi everybody, > > after an update this morning to dovecot 2.3.7 we cannot connect to our > shared mailboxes anymore. The error message as issued in Thunderbird: > > "Character not allowed in mailbox name '.'" > > I didn't change anything about the configuration. Output of dovecot -n: > > [...] > namespace { > list = children > location = maildir:%%h:INDEX=%h/shared/%%u > prefix = Shared/%%u/ > separator = / > subscriptions = yes > type = shared > } > [...] > > Any help would be appreciated, thank you very much. > > Lothar Schilling > > > > >

3 2

Re: Dovecot 2.3.6 on Solaris10: build issues, segfaults
by Joseph Tam 23 Jul '19

23 Jul '19

Looking further into this segfault at settings-parser.c:setting_copy():1519 *dest_size = *src_size; *src_size points to type size_t (typedef unsigned long), a 4-byte aligned value consistent with a 32-bit build. This is mismatched with declared type (gdb) whatis src_size type = const uoff_t * (gdb) whatis uoff_t type = unsigned long long (gdb) p sizeof(uoff_t) $1 = 8 resulting in the segfault when *src_size is dereferened. The implied condition of this code segment is typeof(uoff_t)==typeof(size_t) which is clearly not the case. I'm not sure how/if uoff_t is defined, but configure reports checking for uoff_t... no checking type of off_t... long long The latter is weird, because if I compile and run using the same compiler flags #include <stdio.h> int main(void) { printf("%d %d\n",sizeof(long long),sizeof(off_t)); } the output is "8 4". Joseph Tam <jtam.home(a)gmail.com>

2 2

Dovecot with MySQL over SSL.
by Reio Remma 22 Jul '19

22 Jul '19

Hello! I'm attempting to get Dovecot working with MySQL user database on another machine. I can connect to the MySQL (5.7.26) instance with SSL enabled: mysql -h db.mrst.ee --ssl-ca=/etc/dovecot/ca.pem --ssl-cert=/etc/dovecot/client-cert.pem --ssl-key=/etc/dovecot/client-key.pem --ssl-cipher=DHE-RSA-AES256-SHA -u vmail -p However if I use the same values in dovecot-sql.conf.ext, I get the following error: Jul 19 00:20:18 turin dovecot: master: Dovecot v2.3.7 (494d20bdc) starting up for imap, lmtp, sieve (core dumps disabled) Jul 19 00:20:18 turin dovecot: auth-worker(82996): Error: mysql(db.mrst.ee): Connect failed to database (vmail): SSL connection error: protocol version mismatch - waiting for 1 seconds before retry Jul 19 00:20:18 turin dovecot: auth-worker(82996): Error: mysql(db.mrst.ee): Connect failed to database (vmail): SSL connection error: protocol version mismatch - waiting for 1 seconds before retry Jul 19 00:20:19 turin dovecot: auth-worker(82996): Error: mysql(db.mrst.ee): Connect failed to database (vmail): Connections using insecure transport are prohibited while --require_secure_transport=ON. - waiting for 5 seconds before retry Jul 19 00:20:19 turin dovecot: auth-worker(82996): Error: mysql(db.mrst.ee): Connect failed to database (vmail): Connections using insecure transport are prohibited while --require_secure_transport=ON. - waiting for 5 seconds before retry Database connection string: connect = host=db.mrst.ee dbname=vmail user=vmail password=stuff \ ssl_ca=/etc/dovecot/ca.pem \ ssl_cert=/etc/dovecot/client-cert.pem \ ssl_key=/etc/dovecot/client-key.pem \ ssl_cipher=DHE-RSA-AES256-SHA If I leave the ssl_cipher unset, I get: Jul 19 00:23:41 turin dovecot: auth-worker(83069): Error: mysql(db.mrst.ee): Connect failed to database (vmail): SSL connection error: Failed to set ciphers to use - waiting for 1 seconds before retry Any ideas? Thanks! Reio

4 9

index worker 2.3.7 undefined symbol errors
by Dirk Koopman 22 Jul '19

22 Jul '19

Suddenly I am getting undefined symbol errors having updated to 2.3.7. Jul 18 01:02:49 localhost dovecot: indexer-worker: Error: User <someone> lookup failed: Couldn't load required plugin /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so: undefined symbol: command_hook_register (I think I fixed this one by a config change) But I am left with this: Jul 19 14:09:52 localhost dovecot: indexer-worker: Error: User <someone> lookup failed: Couldn't load required plugin /usr/lib/dovecot/modules/lib90_sieve_plugin.so: dlopen() faile: /usr/lib/dovecot/modules/lib90_sieve_plugin.so: undefined symbol: mail_deliver_ctx_get_log_var_expand_table My (redacted) dovecot -n: # 2.3.7 (494d20bdc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7 (5a4e63b5) # OS: Linux 4.15.0-54-generic x86_64 Ubuntu 18.04.2 LTS # Hostname: localhost auth_verbose = yes first_valid_uid = 2000 imapc_features = rfc822.size fetch-headers imapc_host = mailman.tobit.co.uk imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = no imapc_user = %n lda_mailbox_autocreate = yes mail_attachment_dir = %h/attachments mail_attachment_min_size = 64 k mail_location = mdbox:%h:INBOX=%h/mail mail_plugins = mail_log notify replication fts fts_lucene sieve mail_prefetch_count = 20 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = # hidden driver = sql } plugin { fts = lucene fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_autoindex_exclude3 = \Deleted fts_lucene = whitespace_chars=@. imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * mail_log_events = delete undelete expunge copy save mailbox_delete mailbox_rename mailbox_create mail_log_fields = uid box msgid size sieve = file:~/sieve;active=~/.dovecot.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_dir = /tmp sieve_trace_level = matching } postmaster_address = postmaster(a)tobit.co.uk protocols = imap lmtp sieve service auth { unix_listener exim-auth { group = Debian-exim mode = 0660 user = Debian-exim } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = # hidden } unix_listener lmtp { mode = 0666 } user = vmail } service managesieve-login { inet_listener sieve { port = # hidden } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 100 } ssl_cert = # hidden ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = # hidden driver = sql } protocol imap { mail_plugins = mail_log notify replication fts fts_lucene imap_sieve } protocol lmtp { mail_plugins = fts fts_lucene sieve }

2 5

Pure-FTPd’s SNI daemon
by Felipe Gasper 22 Jul '19

22 Jul '19

Hello, Pure-FTPd just adopted an interesting approach to SNI support: a separate daemon that serves cert chain and key for a given hostname. What would be involved in having Dovecot support this daemon? It would alleviate the need right now for a very large “sni.conf” file on hosts that serve thousands of domains. Thank you! -Felipe Gasper Mississauga, ON

1 0

Dovecot eBook
by Peter Fraser 22 Jul '19

22 Jul '19

Hi Timo I have a strange question. I bought the Dovecot Book off Amazon. I can easily prove it with a picture and/or my receipt off Amazon. I still have it o my library but I don’t like to travel around with it. Is there a way for me to get a PDF copy? I just checked Amazon and there is still no PDF version available there. Regards SI Sent from Mail for Windows 10

7 8

Character not allowed in mailbox name
by Lothar Schilling 22 Jul '19

22 Jul '19

Hi everybody, after an update this morning to dovecot 2.3.7 we cannot connect to our shared mailboxes anymore. The error message as issued in Thunderbird: "Character not allowed in mailbox name '.'" I didn't change anything about the configuration. Output of dovecot -n: [...] namespace { list = children location = maildir:%%h:INDEX=%h/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } [...] Any help would be appreciated, thank you very much. Lothar Schilling

2 1

Replication issue 2.3.7
by Reio Remma 20 Jul '19

20 Jul '19

Hello! I noticed these in the logs since upgrading from 2.3.6. to 2.3.7: Jul 13 11:52:10 turin dovecot: doveadm: Error: dsync-remote(reio(a)mrstuudio.ee)</1ekE7qbKV2gYwAAsNnMGQ>: Error: Exporting mailbox INBOX failed: Mailbox attribute vendor/vendor.dovecot/pvt/server/sieve/files/MR lookup failed: Mailbox attributes not enabled Jul 13 11:52:11 turin dovecot: doveadm: Error: dsync-remote(reio(a)mrstuudio.ee)<MVs/K7ubKV2oYwAAsNnMGQ>: Error: Exporting mailbox INBOX failed: Mailbox attribute vendor/vendor.dovecot/pvt/server/sieve/files/MR lookup failed: Mailbox attributes not enabled After turning on mailbox attributes these errors went away: mail_attribute_dict = file:~/Maildir/dovecot-attributes protocol imap { imap_metadata = yes } But now the errors are replaced with (when deleting mail): Jul 13 12:04:32 turin dovecot: imap(reio(a)mrstuudio.ee): Warning: /home/vmail/mrstuudio.ee/reio/Maildir/dovecot-uidlist: Duplicate file entry at line 2: 1563008644.M18534P25946.orc.mrstuudio.ee,S=4180,W=4262 (uid 23030 -> 23031) - retrying by re-reading from beginning Jul 13 12:04:32 turin dovecot: imap(reio(a)mrstuudio.ee): Warning: Maildir /home/vmail/mrstuudio.ee/reio/Maildir: Expunged message reappeared, giving a new UID (old uid=23030, file=1563008644.M18534P25946.orc.mrstuudio.ee,S=4180,W=4262:2,S) The mail message reappears on the other side of dsync and eventually I end up with 3 identical messages in trash after I've deleted them on both sides. Thanks for any advice, Reio

5 7

Address family not supported by protocol
by Elias Hamaz 19 Jul '19

19 Jul '19

2 1

Address family not supported by protocol
by Elias Hamaz 19 Jul '19

19 Jul '19

Hello, I am trying to run Dovecot on Arch Linux ARM on a Raspberry Pi, and there is an error when calling dovecot. Related information is listed below. Does anybody know if this is a code issue or a configuration issue? Regards, Elias Hamaz $ sudo dovecot: Error: socket() failed: Address family not supported by protocol Error: service(pop3-login): listen(::, 110) failed: Address family not supported by protocol Error: socket() failed: Address family not supported by protocol Error: service(pop3-login): listen(::, 995) failed: Address family not supported by protocol Error: socket() failed: Address family not supported by protocol Error: service(imap-login): listen(::, 143) failed: Address family not supported by protocol Error: socket() failed: Address family not supported by protocol Error: service(imap-login): listen(::, 993) failed: Address family not supported by protocol Fatal: Failed to start listeners $ df -Th: Filesystem Type Size Used Avail Use% Mounted on /dev/root ext4 15G 13G 1.1G 93% / devtmpfs devtmpfs 211M 0 211M 0% /dev tmpfs tmpfs 215M 0 215M 0% /dev/shm tmpfs tmpfs 215M 280K 215M 1% /run tmpfs tmpfs 215M 0 215M 0% /sys/fs/cgroup tmpfs tmpfs 215M 0 215M 0% /tmp /dev/mmcblk0p1 vfat 90M 49M 41M 55% /boot tmpfs tmpfs 43M 0 43M 0% /run/user/1000 $ lscpu: Architecture: armv6l Byte Order: Little Endian CPU(s): 1 On-line CPU(s) list: 0 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 1 Vendor ID: ARM Model: 7 Model name: ARM1176 Stepping: r0p7 CPU max MHz: 700.0000 CPU min MHz: 700.0000 BogoMIPS: 697.95 Flags: half thumb fastmult vfp edsp java tls $ dovecot -n: # 2.3.7 (494d20bdc): /etc/dovecot/dovecot.conf # OS: Linux 4.14.50-2-ARCH armv6l # Hostname: alarmpi doveconf: Error: t_readlink(/var/run/dovecot/dovecot.conf) failed: readlink() failed: No such file or directory namespace { inbox = yes location = mailbox { special_use = \Drafts name = Drafts } mailbox { special_use = \Junk name = Junk } mailbox { special_use = \Sent name = Sent } mailbox { special_use = \Sent name = Sent Messages } mailbox { special_use = \Trash name = Trash } prefix = name = inbox } passdb { driver = pam } service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol lmtp { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } } protocol !indexer-worker { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } } protocol lda { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } } protocol imap { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } } protocol pop3 { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } } protocol submission { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict-async { name = dict-async } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap-hibernate { name = imap-hibernate } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service imap-master { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service old-stats-mail { name = old-stats } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service stats-reader { name = stats } service submission { name = submission-login } service login/submission { name = submission } }

1 0

Re: Dovecot 2.3.0 TLS
by Alexandre Urban 19 Jul '19

19 Jul '19

Hello, I don't know who will read this message, but I found this thread: https://www.mail-archive.com/search?l=dovecot@dovecot.org&q=subject:%22Dove… And I'm expected the same issue, I will try to explain to you (english is not my native language, sorry) Since Buster update, so Dovecot update too, I'm not able to connect to my mail server from my iOS mail client (12.2) Thunderbird just work fine. Here is my configuration: Debian Buster (amd64) Dovecot: 2.3.4.1 Postfix : 3.4.5 OpenSSL: 1.1.1c Dovecot configuration file: ssl_min_protocol = TLSv1.2 (I tried different version) When I tried to connect with command line: openssl s_client -showcerts -connect server:993 No client certificate CA names sent Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 2322 bytes and written 392 bytes Verification error: unable to verify the first certificate --- New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 Server public key is 2048 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 21 (unable to verify the first certificate) When I tried to connect with command line: openssl s_client -showcerts -no_tls1_3 -connect server:993 No client certificate CA names sent Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 2423 bytes and written 310 bytes Verification error: unable to verify the first certificate --- New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 I think the "Secure Renegotiation IS NOT supported" with tls 1.3 could be an issue, but I don't what to do to fix the issue ? Could you help me ? Let me know if you need more informations. Thank you. Regards, Alex

2 1

Panic alert
by marvin alcantara 19 Jul '19

19 Jul '19

Hi, I am getting a panic alert when sending an email in my box. Below is the backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9f3de) [0x7fa90b5543de] -> /usr/lib64/dovecot/libdovecot.so.0(+0x9f4be) [0x7fa90b5544be] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa90b4e577c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_search_args_unref+0xa2) [0x7fa90b827c82] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_search_result_free+0x7a) [0x7fa90b84029a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_thread_init+0x128) [0x7fa90b8b2e88] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_storage_search_init+0x471) [0x7fa90b8a78a1] -> /usr/lib64/dovecot/lib20_virtual_plugin.so(virtual_search_init+0x23) [0x7fa90a0ac3d3] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_search_result_update_flags+0x11f) [0x7fa90b8a4f8f] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_sync_search_results_update+0x5b) [0x7fa90b8afe9b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(index_mailbox_sync_deinit+0x1a4) [0x7fa90b8b0724] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xaa03f) [0x7fa90b89103f] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_deinit+0x38) [0x7fa90b830cb8] -> dovecot/imap(+0x27111) [0x564ec6db6111] -> dovecot/imap(imap_sync_more+0x10d) [0x564ec6db6ccd] -> dovecot/imap(+0x28356) [0x564ec6db7356] -> dovecot/imap(cmd_sync_delayed+0x1f4) [0x564ec6db7814] -> dovecot/imap(client_handle_input+0x218) [0x564ec6daa968] -> dovecot/imap(client_input+0x85) [0x564ec6daae05] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7fa90b569cd2] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x10f) [0x7fa90b56b3bf] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7fa90b569d6c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa90b569f28] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa90b4effa3] -> dovecot/imap(main+0x333) [0x564ec6d9d2e3] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa90b10a445] -> dovecot/imap(+0xe476) [0x564ec6d9d476] Dovecot version: 2.2.33.2 I would like to ask help on what is happening here. I found in the code that the trigger is in plugins/virtual/virtual-sync.c virtual_sync() call. When does this actually being called? Thank you very much! Regards,Marvin Alcantara

1 0

auth: Warning: Event 0x1280fe20 leaked
by John Doe Vecot 18 Jul '19

18 Jul '19

Lately I get some errors in my logfile, saying: dovecot[72337]: auth: Warning: Event 0x1280fe20 leaked (parent=0x0): auth-client-connection.c:338: 1 Time(s) Can you tell me what is going wrong here? Thanks.

4 4

Dovecot release v2.3.7
by Aki Tuomi 17 Jul '19

17 Jul '19

Hi! We are pleased to release Dovecot release v2.3.7. Tarball is available at https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz.sig Binary packages are available at https://repo.dovecot.org/ Changes ------- * fts-solr: Removed break-imap-search parameter + Added more events for the new statistics, see https://doc.dovecot.org/admin_manual/list_of_events/ + mail-lua: Add IMAP metadata accessors, see https://doc.dovecot.org/admin_manual/lua/ + Add event exporters that allow exporting raw events to log files and external systems, see https://doc.dovecot.org/configuration_manual/event_export/ + SNIPPET is now PREVIEW and size has been increased to 200 characters. + Add body option to fts_enforced. This triggers building FTS index only on body search, and an error using FTS index fails the search rather than reads through all the mails. - Submission/LMTP: Fixed crash when domain argument is invalid in a second EHLO/LHLO command. - Copying/moving mails using Maildir format loses IMAP keywords in the destination if the mail also has no system flags. - mail_attachment_detection_options=add-flags-on-save caused email body to be unnecessarily opened when FETCHing mail headers that were already cached. - mail attachment detection keywords not saved with maildir. - dovecot.index.cache may have grown excessively large in some situations. This happened especially when using autoexpunging with lazy_expunge folders. Also with mdbox format in general the cache file wasn't recreated as often as it should have. - Autoexpunged mails weren't immediately deleted from the disk. Instead, the deletion from disk happened the next time the folder was opened. This could have caused unnecessary delays if the opening was done by an interactive IMAP session. - Dovecot's TCP connections sometimes add extra 40ms latency due to not enabling TCP_NODELAY. HTTP and SMTP/LMTP connections weren't affected, but everything else was. This delay wasn't always visible - only in some situations with some message/packet sizes. - imapc: Fix various crash conditions - Dovecot builds were not always reproducible. - login-proxy: With shutdown_clients=no after config reload the existing connections could no longer be listed or kicked with doveadm. - "doveadm proxy kick" with -f parameter caused a crash in some situations. - Auth policy can cause segmentation fault crash during auth process shutdown if all auth requests have not been finished. - Fix various minor bugs leading into incorrect behaviour in mailbox list index handling. These rarely caused noticeable problems. - LDAP auth: Iteration accesses freed memory, possibly crashing auth-worker - local_name { .. } filter in dovecot.conf does not correctly support multiple names and wildcards were matched incorrectly. - replicator: dsync assert-crashes if it can't connect to remote TCP server. - config: Memory leak in config process when ssl_dh setting wasn't set and there was no ssl-parameters.dat file. This caused config process to die once in a while with "out of memory". --- Aki Tuomi Open-Xchange oy

5 16

pigeonhole question: filtering on delivered-to in case of fetchmail
by Trever L. Adams 17 Jul '19

17 Jul '19

So, one of the problems I am seeing is that people are trying to fake users into revealing information by sending from an outside domain but with an internal reply to address and claiming to be administration, IT or what not. I can set up something that will reject if from is outside the domain by reply to is internal. The problem is in some setups, there are fetchmail setups. I do not want to reject these with a message. Which is what I am currently doing for the others. Maybe I should discard them all without rejecting. However, my question is this: Since such fetchmail messages will usually end up with two (at least two?!?) Delivered-To headers, one for the fetchmail delivery and one for the original target address's/system's delivery is it possible to do something like this and have it work? require ["fileinto", "regex","reject"]; if address :regex "Reply-To" ".*@<%= @name -%>" { if not address :regex "From" ".*@<%= @name -%>" { if not header :regex "Delivered-To" " .*@<%= @name -%> { reject "We do not allow emails from outside our system to give Reply-To into our system!"; stop; } } } Please, not the <% =@name -%> is just that this is from a puppet module I use to maintain these systems. It is the domain name for the mail system. An example would be .*(a)middleearth.sapphiresunday.org here. Thank you for any help in figuring this out. The reason I want a reject in the case of non-fetchmail email is to let users know if they try to do it (as many have multiple email accounts) and may try it. But in fetchmail cases, no need to leak to the outside world that users are doing fetchmail and what their account is in the other system. Thank you. Trever

2 3

Dovecot v2.3.7 - TLS/SSL issue
by Christos Chatzaras 17 Jul '19

17 Jul '19

I use a helpdesk system that connects to dovecot using POP3 with SSL enabled to fetch the e-mails. After I upgrade to v2.3.7 the helpdesk randomly (some times it works, some times not) doesn't fetch the e-mails. If I configure the e-mail accounts with SSL/TLS disabled then it works. Any idea about this?

2 1

2.3.7 slower than 2.3.6?
by Ralf Hildebrandt 17 Jul '19

17 Jul '19

We're using dovecot (via LMTP) as a backup for all incoming mail. I upgraded from 2.3.6 to 2.3.7 on the 12th: 2019-07-12 14:35:44 upgrade dovecot-imapd:amd64 2:2.3.6-2~bionic 2:2.3.7-8~bionic and it seems that 2.3.7 is slower than 2.3.6 -- mail to the backup IMAP box is suddenly taking quite some time to get delivered and is piling up in the queue. doveconf -n: ============ default_vsz_limit = 2 G lmtp_user_concurrency_limit = 10000 mail_attachment_dir = /home/copymail/attachments mail_fsync = never mail_location = mdbox:~/mdbox mail_plugins = zlib fts mdbox_rotate_size = 128 M ... namespaces ... plugin { fts_autoindex = yes fts_languages = de,en zlib_save = gz zlib_save_level = 5 } protocols = " imap lmtp" userdb { args = username_format=%u /etc/dovecot/passwd driver = passwd-file } protocol lmtp { mail_plugins = zlib fts } -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 5

Auto-duplicate mailstore?
by ＠lbutlr 16 Jul '19

16 Jul '19

I am looking for something that is similar to replication, but without a second server. I currently have a system setup using postfix bcc to write out copies of all email, but what I would rather do is just have all the mail written to /usr/local/virtual/%u/Maildir and also to /backup/usr/local/virtual/%u/Maildir (for example) as this will make restoring messages much simpler. -- Gods don't like people not doing much work. People who aren't busy all the time might start to think.

1 2

Unexpected result from LIST EXTENDED command
by Emil Kalchev 16 Jul '19

16 Jul '19

I am executing this command below to dovecot-2.3.5-6.cp1178.x86_64 server Notice that some status responses are missing (For folders INBOX.Archive, INBOX.spam.&-BD0EOQQ9BDkEPQ-). I wonder If this is a bug or working as expected In rfc5819 there is this: If the server runs into unexpected problems while trying to look up the STATUS information, it MAY drop the corresponding STATUS reply. In such a situation, the LIST command would still return a tagged OK reply. May be that is the reason for this response? Is it possible to find more details in server logs why STATUS is missing? C: B00000007 LIST "" "INBOX.*" RETURN (SUBSCRIBED CHILDREN STATUS (HIGHESTMODSEQ)) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.gfgfee888 S: * STATUS INBOX.gfgfee888 (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.99999999 S: * STATUS INBOX.99999999 (HIGHESTMODSEQ 3) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.yyyyyyyyyyyyy S: * STATUS INBOX.yyyyyyyyyyyyy (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.y S: * STATUS INBOX.y (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.fgfgfg S: * STATUS INBOX.fgfgfg (HIGHESTMODSEQ 61) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Mailspring S: * STATUS INBOX.Mailspring (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Mailspring.Snoozed S: * STATUS INBOX.Mailspring.Snoozed (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.fgfgf S: * STATUS INBOX.fgfgf (HIGHESTMODSEQ 3) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.gggg34 S: * STATUS INBOX.gggg34 (HIGHESTMODSEQ 5) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.gggg S: * STATUS INBOX.gggg (HIGHESTMODSEQ 2) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.gggg.hhhh S: * STATUS INBOX.gggg.hhhh (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.mkmkm S: * STATUS INBOX.mkmkm (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.sdfsf S: * STATUS INBOX.sdfsf (HIGHESTMODSEQ 1) S: * LIST (\HasChildren \UnMarked) "." INBOX.sdfsf.sdfsdfsdfsd S: * STATUS INBOX.sdfsf.sdfsdfsdfsd (HIGHESTMODSEQ 1) S: * LIST (\HasNoChildren \UnMarked) "." INBOX.sdfsf.sdfsdfsdfsd.zzzz S: * STATUS INBOX.sdfsf.sdfsdfsdfsd.zzzz (HIGHESTMODSEQ 2263) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.uuyy S: * STATUS INBOX.uuyy (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.uuyy.sdfsdf S: * STATUS INBOX.uuyy.sdfsdf (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf.asdasdasd" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf.asdasdasd" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.uuyy.fdfgdfg S: * STATUS INBOX.uuyy.fdfgdfg (HIGHESTMODSEQ 2) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.vvvcv S: * STATUS INBOX.vvvcv (HIGHESTMODSEQ 2627) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.hallo S: * STATUS INBOX.hallo (HIGHESTMODSEQ 96) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.New Folder" S: * STATUS "INBOX.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.omgfolder" S: * STATUS "INBOX.New Folder.omgfolder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.testfolder" S: * STATUS "INBOX.New Folder.testfolder" (HIGHESTMODSEQ 1) S: * LIST (\HasNoChildren \UnMarked) "." "INBOX.New Folder.hkhk" S: * STATUS "INBOX.New Folder.hkhk" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.zzz" S: * STATUS "INBOX.New Folder.zzz" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.aaa" S: * STATUS "INBOX.New Folder.aaa" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.kgkgkk" S: * STATUS "INBOX.New Folder.kgkgkk" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.gggfff" S: * STATUS "INBOX.New Folder.gggfff" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.New Folder" S: * STATUS "INBOX.New Folder.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.hhhh" S: * STATUS "INBOX.New Folder.hhhh" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.nnnn" S: * STATUS "INBOX.New Folder.nnnn" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.asdasd S: * STATUS INBOX.asdasd (HIGHESTMODSEQ 23) S: * LIST (\HasNoChildren \UnMarked) "." INBOX.asdasd.gdfgdfgdfg S: * STATUS INBOX.asdasd.gdfgdfgdfg (HIGHESTMODSEQ 10) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.spam S: * STATUS INBOX.spam (HIGHESTMODSEQ 44) S: * LIST (\HasChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ- S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg S: * STATUS INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.New Folder" S: * STATUS "INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa S: * STATUS INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa.aaa S: * STATUS INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa.aaa (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Sent S: * STATUS INBOX.Sent (HIGHESTMODSEQ 1353) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Trash S: * STATUS INBOX.Trash (HIGHESTMODSEQ 5823) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk S: * STATUS INBOX.Junk (HIGHESTMODSEQ 15) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk.asdasd S: * STATUS INBOX.Junk.asdasd (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk.asdasd.rgthfgh S: * STATUS INBOX.Junk.asdasd.rgthfgh (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Junk.asdasd.rgthfgh.Mailspring S: * STATUS INBOX.Junk.asdasd.rgthfgh.Mailspring (HIGHESTMODSEQ 28) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk.asdasd.asdasdasd S: * STATUS INBOX.Junk.asdasd.asdasdasd (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Junk.asdasd.asdasdasd.dfgdfg S: * STATUS INBOX.Junk.asdasd.asdasdasd.dfgdfg (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Drafts S: * STATUS INBOX.Drafts (HIGHESTMODSEQ 1091) S: * LIST (\HasNoChildren \UnMarked) "." INBOX.Archive S: B00000007 OK List completed (0.004 + 0.000 + 0.003 secs).

3 4

Replication Upgrade 2.2 -> 2.3 Question
by Cassidy B. Larson 16 Jul '19

16 Jul '19

Finally getting around to upgrading my setup to 2.3.x. All of our front end proxies are fine on 2.3 connecting to a 2.2 backend storage pair. So the question is, if I stage the upgrade, will it break replication between the two backend storage servers? My thought was to upgrade one to 2.3, and keep the other on 2.2 for a couple of days? If so, can 2.2 replicate to 2.3, and 2.3 replicate to 2.2 without any issue? Or am I going to have to go all-in on 2.3 at once on both? Heh. Thanks, -c

1 0

Listing expunged messages
by Mike 15 Jul '19

15 Jul '19

Hello, Following on my last question about broken mdboxes, removing dovecot.map.log and force-resync does work. My last problem now however is that after doing this, messages which were 'deleted' previously, come back in the inbox as new messages. I have the original mailboxes and can subscribe to the expunged folders and everything in EXPUNGED/INBOX on the original, is appearing in INBOX on this migrated system. I would want to return everything to EXPUNDED/INBOX so my users don't see old dead resurrected messages. Is there a procedure to identify these? Thanks. Mike-

1 0

Weird response to LIST EXTENDED
by Emil Kalchev 15 Jul '19

15 Jul '19

Hello, I was discussing weird response from Dovecot on Mailkit (this is client IMAP C# library) github page. We concluded that this is a bug in Dovecot server but I wanted to double check with you. Here are the details about the issue https://github.com/jstedfast/MailKit/issues/879 Sent with OfficeSuite Mail for Windows PC www.officesuitenow.com<https://www.officesuitenow.com>

1 0

Authdb NSS module
by Eugene 15 Jul '19

15 Jul '19

Hello! Upgrading manual tells that authdb [NSS module was removed][1] some time ago. [1]: https://wiki2.dovecot.org/Upgrading/2.3#line-100 > userdb nss was removed. Use userdb passwd instead. Can this change be reverted? I'd like to use only libnss_sss.so.2 as dovecot userdb source. It's also essential for me to enable files backend in nsswitch.conf so the system could use local user db. At the same time dovecot must not see local users at all. Authdb NSS module could help me there. The other solution would be to use another instance of nsswitch.conf for dovecot authdb passwd module. Is it possible? Thanks! -- Eugene Bright IT engineer Tel: + 79257289622

3 3

Error: o_stream_send_istream and Disconnected in APPEND
by Alessio Cecchi 15 Jul '19

15 Jul '19

Hi, I'm running some Dovecot servers configured with LVS + Director + Backend + NFS and version 2.2.36.3 (a7d78f5a2). In the last days I see an increased number of these error: Error: o_stream_send_istream(/nfs/mail/company.com/info/Maildir/.Sent/tmp/15627713… failed: Broken pipe always with action "Disconnected in APPEND", when users try to upload a message in Sent or Drafts. Here the logs: Director: Jul 10 14:53:06 imap-login: Info: proxy(info(a)company.com): started proxying to 10.0.0.20:143: user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, lport=993, TLS, session=<eA6XKFONwP1f5Sh4> Jul 10 15:09:56 imap-login: Info: proxy(info(a)company.com): disconnecting 84.1.2.3 (Disconnected by client: EOF(0s idle, in=648413, out=6602)): user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, lport=993, TLS, session=<eA6XKFONwP1f5Sh4> Backend: Jul 10 14:53:06 pop01 dovecot: imap-login: ID sent: x-session-id=eA6XKFONwP1f5Sh4, x-originating-ip=84.1.2.3, x-originating-port=64960, x-connected-ip=195.1.2.3, x-connected-port=993, x-proxy-ttl=4: user=<>, rip=84.1.2.3, lip=195.1.2.3, secured, session=<eA6XKFONwP1f5Sh4> Jul 10 14:53:06 pop01 dovecot: imap-login: Login: user=<info(a)company.com>, method=PLAIN, rip=84.1.2.3, lip=195.1.2.3, mpid=9151, secured, session=<eA6XKFONwP1f5Sh4> Jul 10 14:53:07 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: ID sent: name=Thunderbird, version=60.7.2 Jul 10 15:09:56 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: Error: o_stream_send_istream(/nfs/mail/company.com/info/Maildir/.Sent/tmp/15627713… failed: Broken pipe Jul 10 15:09:57 pop01 dovecot: imap(info(a)company.com) session=<eA6XKFONwP1f5Sh4>: Disconnected in APPEND (1 msgs, 48 secs, 0/2657336 bytes) in=886661 out=26654 del=1 expu=0 trash=0 I can't understand if is a network problem (Firewall? Load Balancer? Switch?) or a users LAN problem. Or others. When this happen user see an error message like "Unable to save email in Sent folder". Any suggestions? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 3

Sieve problem with duplicate and fileinto in the same set of rules
by Gianluca Scaglia 15 Jul '19

15 Jul '19

Hi there, on my mail server (postfix, dovecot 2.2.27 in Debian 9) I have an automatic forwarding (with sender_bcc_maps in Postfix) for all the emails sent in smtp from the same server, that are then put in the Sent folder with a sieve rule. In this way, however, when a user sends an e-mail to himself, both copies end up in the Sent folder and it's not good. To resolve, I tried using the Sieve "duplicate" extension along with "fileinto" but I can't get it to work. I then activated also the "editheader" extension to better understand what the problem was and I get these results. Sieve rules without using "fileinto:" require ["duplicate", "editheader"]; if duplicate { addheader "x-sieve-test1" "duplicate"; } if allof (header :regex ["from"] [".*info(a)example\.com"], header :regex ["x-sieve-test1"] [".*duplicate"]) { addheader "x-sieve-test2" "confirm duplicate"; stop; } elsif header :regex ["from"] [".*info(a)example\.com"] { addheader "x-sieve-test3" "not duplicate"; stop; } and I get these headers in the first mail (in INBOX): X-Sieve-Test2: confirm duplicate X-Sieve-Test1: duplicate and in the second email (always in INBOX) this other: X-Sieve-Test3: not duplicate Everything works perfectly. When I add the fileinto statement to the last if as below require ["duplicate", "editheader"]; if duplicate { addheader "x-sieve-test1" "duplicate"; } if allof (header :regex ["from"] [".*info(a)example\.com"], header :regex ["x-sieve-test1"] [".*duplicate"]) { addheader "x-sieve-test2" "confirm duplicate"; stop; } elsif header :regex ["from"] [".*info(a)example\.com"] { addheader "x-sieve-test3" "not duplicate"; fileinto "Sent"; stop; } I get a completely incorrect result, both mails in the Sent folder with this header line: X-Sieve-Test3: not duplicate in practice it no longer detects the duplicate email. If I use "discard" rather than "fileinto" it works fine and if I put "fileinto" in the first "if" ("if duplicate") it does't work. I did hundreds of tests but the "duplicate" extension stops working in the presence of the fileinto statement (there are exceptions: if I put the "fileinto" in the second "if" - that is in the middle one instead of the last one - it works fine, but it's not that that I want). Any idea how to solve the problem or how to get the same result (one mail in Sent folder and one in INBOX, when a user sends an e-mail to himself) without using the "duplicate" extension? Thanks a lot, Luca

2 1

Error since Dovecot v2.3.7
by Paul Hecker 15 Jul '19

15 Jul '19

Hi, since upgrading to Dovecot 2.3.7 I get the following new errors: 2019-07-15 09:10:52 mail dovecot: imap(paul(a)iwascoding.com)<32484><goWh8rKNSNZfqJ9R>: Error: file_lock_free(): Unexpectedly failed to retry locking /var/spool/mail/iwascoding/paul/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-vsize.lock: fcntl(/var/spool/mail/iwascoding/paul/mdbox/mailboxes/INBOX/dbox-Mails/dovecot-vsize.lock, write-lock, F_SETLK) locking failed: No such file or directory It seems that this is related to moving or deleting mails (IMAP). The file at least sometimes exists, the folder exists. You find my doveconf below. Thanks, Paul # 2.3.7 (494d20bdc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.7 (5a4e63b5) # OS: Linux 4.9.0-9-amd64 x86_64 Debian 9.9 # Hostname: mail.iwascoding.com auth_cache_size = 10 M auth_cache_ttl = 2 hours auth_mechanisms = plain login default_vsz_limit = 1 G first_valid_gid = 8 first_valid_uid = 8 hostname = mail.iwascoding.com imap_capability = +SPECIAL-USE last_valid_gid = 8 last_valid_uid = 8 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_attachment_detection_options = add-flags-on-save mail_attribute_dict = file:/var/mail/iwascoding/%n/dovecot-attributes mail_gid = 8 mail_home = /var/mail/iwascoding/%n mail_location = mdbox:~/mdbox mail_plugins = quota listescape fts fts_xapian mail_prefetch_count = 200 mail_uid = 8 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox archive { auto = no special_use = \Archive } mailbox drafts { auto = no special_use = \Drafts } mailbox junk { auto = no special_use = \Junk } mailbox sent { auto = no special_use = \Sent } mailbox trash { auto = no special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = xapian fts_autoindex = yes fts_autoindex_exclude = \Junk fts_autoindex_exclude2 = \Trash fts_autoindex_exclude3 = \Drafts fts_enforced = yes fts_xapian = partial=2 full=20 plugin = fts fts_xapian quota = count:User quota quota_grace = 10%% quota_rule = *:storage=200M quota_vsizes = yes recipient_delimiter = + sieve = /var/mail/iwascoding/%n/.dovecot.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = /var/mail/iwascoding/%n/sieve sieve_extensions = +spamtest +spamtestplus sieve_global_dir = /var/lib/dovecot/sieve/global sieve_spamtest_max_value = 200 sieve_spamtest_status_header = X-Spam-Score-Int sieve_spamtest_status_type = score } postmaster_address = postmaster(a)iwascoding.com protocols = imap pop3 lmtp sieve submission quota_full_tempfail = yes service auth { unix_listener auth-client { mode = 0600 user = Debian-exim } unix_listener auth-userdb { user = mail } vsz_limit = 4 G } service imap-login { inet_listener imap { port = 0 } } service indexer-worker { vsz_limit = 2 G } service lmtp { user = mail } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 0 } } service stats { unix_listener stats-writer { user = mail } } service submission-login { inet_listener submission_ssl { port = 465 ssl = yes } } ssl = required ssl_alt_cert = </usr/local/acme/var/iwascoding.com/fullchain.cer ssl_alt_key = # hidden, use -P to show it ssl_cert = </usr/local/acme/var/iwascoding.com_ecc/fullchain.cer ssl_cipher_list = ALL:HIGH:!SSLv2:!SSLv3:!kRSA:!MEDIUM:!LOW:!EXP:!RC4:!MD5:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!PSK:!SRP:!DSS:!SEED:@STRENGTH ssl_client_ca_dir = /etc/ssl/certs ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes submission_client_workarounds = whitespace-before-path submission_max_mail_size = 50 M submission_relay_host = mail.iwascoding.com submission_relay_port = 10025 submission_relay_ssl = smtps userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_fsync = optimized mail_plugins = quota listescape fts fts_xapian sieve } protocol !indexer-worker { mail_vsize_bg_after_count = 100 } protocol lda { mail_fsync = optimized mail_plugins = quota listescape fts fts_xapian sieve } protocol imap { imap_metadata = yes mail_max_userip_connections = 40 mail_plugins = quota listescape fts fts_xapian imap_quota imap_zlib } protocol pop3 { mail_max_userip_connections = 30 }

2 2

broken mdboxes
by Mike 15 Jul '19

15 Jul '19

Hello, I was forced to do an emergency server migration with about 3000 mailboxes and I botched it somewhat. Hoping to find some help here. My symptoms are, users showing apparently deleted mail messages, some replicated mail, and a time period of mail that is just 'missing'. The source server is centos 6.10 with dovecot 2.2.5, and the destination is ubuntu 16 / dovecot 2.2.22. User mailboxes are mdbox format and I did an rsync from the old to the new. My screw up however was that my rsync was not --delete files, and had to be run multiple times over the course of a few weeks due to to bandwidth disparities and the sheer size of the mail spool approaching 700GB (source server had 10mbps. Should have used sneakernet instead). The bottom line however is that I wound up having many extra m.* files in each user storage dir that were not actually present on the source when the final, final sync was made and I went live with the migration server. I know this is my fsckup but now I have to deal with it. I first tried to fix the symptoms with a force-resync on the boxes not realizing the above, and that seemed to make things worse by only showing new mail as of the server migration date. Ugh. New mail is being delivered and this does appear in user mail boxes. But as I said, there seems to be some users with a problem with old mail presumably deleted on the prior server suddenly re-appearing (probbly due to the extraneous m.* mdbox files present). A more pressing problem is that there is simply a time period of mail that is in user mailboxes (on the old server) from before the migration, which is not displaying now for these users on the new server. I have investigated this and hacked on "mdbox-recover.pl" somewhat so that it iterates over all mdbox files for a user and I can extract ALL messages, and see that, yes, the mail really is present in the mdbox files, but pop/imap never shows it. This is puzzling to me. I have a test user that when I check imap, it shows only 21 message in inbox and 1 in sent. But if I extract the messages from the mdbox files, then I actually have 1117 messages. I know I can't just rsync the source server again with the right --delete flag, there has been lots of mail deliveries to the migrated boxes and so I don't want to lose all of that and create yet more problems. But I need to find some way to make these present mail messages appear for my users again. Im thinking it's got to be a flag of some kind since the data is there it's just ignored. Wondering what you would reccomend? Thanks. Mike-

2 3

Unable to send the mail but able to receive them
by necktwi 15 Jul '19

15 Jul '19

From my mac mail client I've connected to remote exim+dovecot server. Since few weeks back I was not able to send mail but able to receive. exim in /var/log/messages says: Jul 14 15:22:15 RPi3B exim[3576]: 2019-07-14 20:52:15 dovecot_plain authenticator failed for ([xx.xx.xx.xx]) [yy.yy.yy.yy]: 535 Incorrect authentication data (set_id=necktwi) dovecot in /var/log/dovecot.log says: Jul 14 19:40:27 auth: Debug: client in: AUTH 1 LOGIN service=smtp rip=yy.yy.yy.yy lip=xx.xx.xx.xx nologin resp= (previous base64 data may contain sensitive data) Jul 14 19:40:27 auth: Debug: client passdb out: CONT 1 VXNlcm5hbWU6 Jul 14 19:40:27 auth: Debug: auth client connected (pid=0) Jul 14 19:40:27 auth: Debug: client in: AUTH 1 PLAIN service=smtp secured rip=xx.xx.xx.xx lip=yy.yy.yy.yy nologin resp=aeVja333aQeuZWerdHdpAeRecmezaG9lcw== (previous base64 data may contain sensitive data) Jul 14 19:40:27 auth: Debug: policy(necktwi,xx.xx.xx.xx): Policy check action is continue Jul 14 19:40:27 auth-worker(11866): Debug: pam(necktwi,xx.xx.xx.xx): lookup service=smtp Jul 14 19:40:27 auth-worker(11866): Info: pam(necktwi,xx.xx.xx.xx): pam_authenticate() failed: Authentication failure (Password mismatch?) (given password: xxxxxxx) (log is obfuscated) Is it the problem with dovecot or exim? Should exim contact dovecot prior to sending the mail? I have also tried windows mail client with same error. auth-system.conf.ext: passdb { driver = pam args = %s } userdb { driver = passwd args = blocking=no } ... Necktwi

1 0

Sent messages disappear and don't arrive. (Receiving is ok)
by John 15 Jul '19

15 Jul '19

Hi List, I'm running an email server (Postfix, Dovecot, MySQL) on an RPi. I set it up using this 'howto' https://pestmeester.nl/index.html#11.0. It seemed to work but now sent emails just disappear and are not received. Inward emails are ok. I'm a Linux and Postfix Dovecot learner so please treat me as not being very knowledgeable. I'm not even sure if this is a Postfix or Dovecot issue so I've posted to both. (Nothing from Postfix so far!) I've tried to include what relevant info I thought a helper might need but if I've missed something please let me know. I'll appreciate any help. TIA, John

3 2

Purpose of stats-writer and why doveadm try to open it to dump stats ?
by Jean-Daniel 15 Jul '19

15 Jul '19

Hello, I want to monitor dovecot stats, and so I have an exporter process that run with limited rights. The monitoring user has only access to /var/run/dovecot/stats-reader and it works fine. Doveadm stats dump returns the list of all stats as expected. But each time I run doveadm stats dump, it logs the following error: Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied So what is the purpose of the stats-writer socket, and why doveadm try to open it to simply dump stats ? Is it really something it needs and I should update my user permissions or is it a doveadm bug ?

2 1

Re: Purpose of stats-writer and why doveadm try to open it to dump stats ?
by Jean-Daniel 14 Jul '19

14 Jul '19

> Le 14 juil. 2019 à 11:58, Reio Remma <reio(a)mrstuudio.ee> a écrit : > > On 14.07.2019 10:10, Jean-Daniel via dovecot wrote: >> Hello, >> >> I want to monitor dovecot stats, and so I have an exporter process that run with limited rights. >> The monitoring user has only access to /var/run/dovecot/stats-reader and it works fine. >> Doveadm stats dump returns the list of all stats as expected. >> >> But each time I run doveadm stats dump, it logs the following error: >> >> Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied >> >> So what is the purpose of the stats-writer socket, and why doveadm try to open it to simply dump stats ? >> Is it really something it needs and I should update my user permissions or is it a doveadm bug ? >> > > Hello! > > Depending on the system you're running on, there may be SELinux etc. doing its work. > I know why stats-writer is not accessible, this is on purpose (doveadm run with a uid that explicitly don’t have access to stats-writer). What I want to know is why does doveadm even try to open it to simply dump the stats.

1 0

Getting SSL certificate/key from database
by Yevgeny Kosarzhevsky 14 Jul '19

14 Jul '19

Hello, I am not finding how to read SSL keys/certificates from database. Is this possible? Or only file reads allowed? -- Regards, Yevgeny

2 1

Pigeonhole-0.5.7 fails to build from sources
by Lefteris Tsintjelis 13 Jul '19

13 Jul '19

Latest update fails to build from sources in FreeBSD 12-STABLE --- lda-sieve-plugin.lo --- lda-sieve-plugin.c:82:27: error: variable has incomplete type 'struct smtp_submit_input' struct smtp_submit_input submit_input; ^ lda-sieve-plugin.c:82:9: note: forward declaration of 'struct smtp_submit_input' struct smtp_submit_input submit_input; ^ 1 error generated. *** [lda-sieve-plugin.lo] Error code 1 make[6]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins/lda-sieve --- lda-sieve-log.lo --- mv -f .deps/lda-sieve-log.Tpo .deps/lda-sieve-log.Plo 1 error make[6]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins/lda-sieve *** [all-recursive] Error code 1 make[5]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins 1 error make[5]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src/plugins *** [all-recursive] Error code 1 make[4]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src 1 error make[4]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7/src *** [all-recursive] Error code 1 make[3]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 1 error make[3]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 *** [all] Error code 2 make[2]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 1 error make[2]: stopped in /usr/ports/mail/dovecot-pigeonhole/work/dovecot-2.3-pigeonhole-0.5.7 ===> Compilation failed unexpectedly. Ensure that dovecot-pigeonhole's GSSAPI backend is the same as dovecot2 (make config). Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *** Error code 1 Stop. make[1]: stopped in /usr/ports/mail/dovecot-pigeonhole *** Error code 1 Stop. make: stopped in /usr/ports/mail/dovecot-pigeonhole

2 3

Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS!
by Adam D. Barratt 13 Jul '19

13 Jul '19

On Wed, 2019-07-10 at 12:26 +0100, Arnold Opio Oree wrote: > Understood Adam, > > My thinking is that this is a package released with Debian 10. And so > has everything to do with the release. For clarity here - debian-release is neither a support forum nor a discussion list, nor a means of reporting issues in software contained within a Debian release. Debian has support fora and a bug tracking system to handle such reports and issues, as per https://www.debian.org /support . Having a single list that discusses any issue in any piece of software shipped by Debian cannot possibly scale. Rather, this list is the contact point and team alias for the Release Team, who oversee and manage the release process. It is up to individual package maintainers to triage and deal with issues reported against their packages and then to liaise with us if required. Regards, Adam

2 1

Error: file_ostream.net_set_tcp_nodelay
by Lefteris Tsintjelis 13 Jul '19

13 Jul '19

After upgrading to 2.3.7_1 I get those errors every now and then. Running FreeBSD 12-RELEASE. Any idea what they may be? Jul 13 08:07:11 pop3-login: Error: file_ostream.net_set_tcp_nodelay(, FALSE) failed: Connection reset by peer Jul 13 08:19:10 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer Jul 13 08:24:09 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer Jul 13 08:34:53 imap-login: Error: file_ostream.net_set_tcp_nodelay(, TRUE) failed: Connection reset by peer Lefteris

1 0

Pigeonhole release v0.5.7
by Aki Tuomi 12 Jul '19

12 Jul '19

Hi! We are pleased to release Pigeonhole release v0.5.7. Tarball is available at https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.ta… https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.7.ta… Binary packages are available at https://repo.dovecot.org/ Changes ------- + vacation: Made the subject for the automatic response message produced by the Sieve vacation action configurable. Both the default subject (if the script defines none) and the subject template (e.g. used to add a subject prefix) can be configured. - dsync: dsync-replication does not synchronize Sieve scripts. - imap_sieve_filter: Reduce FILTER=SIEVE verbosity over IMAP connection. - testsuite: Pigeonhole testsuite segfaulted if it was compiled with GCC 9 --- Aki Tuomi Open-Xchange oy

3 4

2.3.7/FreeBSD: Seeing lots of stats-writer errors.
by Larry Rosenman 12 Jul '19

12 Jul '19

A user reported this to me, and I'm seeing it too: Hi all :) After mail/dovecot upgrade to v2.3.7 it seems something went wrong with stat-writer... IMAP and POP3 logs: Jul 12 19:45:34 mail dovecot: imap-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument Jul 12 19:45:34 mail dovecot: imap-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:login), TRUE) failed: Invalid argument Jul 12 19:45:35 mail dovecot: imap: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument Jul 12 19:45:35 mail dovecot: imap: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/auth-master), TRUE) failed: Invalid argument Jul 12 19:45:35 mail dovecot: imap-login: Login: user=<someone(a)domain.tld>, method=PLAIN, rip=192.168.0.28, lip=192.168.0.1, mpid=47811, TLS, session=<aelAen+NKpDAqAAc> Jul 12 19:45:41 mail dovecot: imap(someone(a)domain.tld)<47811><aelAen+NKpDAqAAc>: Connection closed (UID SEARCH finished 6.727 secs ago) in=128 out=2015 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 Jul 12 19:51:21 mail dovecot: pop3-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument Jul 12 19:51:21 mail dovecot: auth: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument Jul 12 19:51:21 mail dovecot: pop3-login: Error: file_ostream.net_set_tcp_nodelay((conn:unix:login), TRUE) failed: Invalid argument Jul 12 19:51:21 mail dovecot: pop3: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/stats-writer), TRUE) failed: Invalid argument Jul 12 19:51:21 mail dovecot: pop3: Error: file_ostream.net_set_tcp_nodelay((conn:unix:/var/run/dovecot/auth-master), TRUE) failed: Invalid argument Jul 12 19:51:21 mail dovecot: pop3-login: Login: user=<someone(a)domain.tld>, method=PLAIN, rip=192.168.0.33, lip=192.168.0.1, mpid=47877, TLS, session=<h+3jjn+NcPrAqAAh> Jul 12 19:51:21 mail dovecot: pop3(someone(a)domain.tld)<47877><h+3jjn+NcPrAqAAh>: Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0 Ideas? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

2 1

new dovecot, selinux Problem ?
by Günther J. Niederwimmer 12 Jul '19

12 Jul '19

Hello List, after the last update I have a selinux "Problem" with dovecot. My system is a centos 7. After a new start from dovecot selinux block a connection. Jul 12 16:24:24 mx01 systemd: Starting Dovecot IMAP/POP3 email server... Jul 12 16:24:54 mx01 systemd: Started Dovecot IMAP/POP3 email server. Jul 12 16:24:54 mx01 dovecot: Warning: Corrected permissions for login directory /var/run/dovecot/token-login Jul 12 16:24:54 mx01 dbus[3008]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper) Jul 12 16:24:55 mx01 dbus[3008]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' Jul 12 16:24:55 mx01 setroubleshoot: SELinux is preventing dovecot from getattr access on the file /proc/sys/fs/suid_dumpable. For complete SELinux messages run: sealert -l c46ae6a7-64c4-49a7-9e3d-477547fb6da8 Jul 12 16:24:55 mx01 python: SELinux is preventing dovecot from getattr access on the file /proc/sys/fs/suid_dumpable.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that dovecot should be allowed getattr access on the suid_dumpable file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'dovecot' --raw | audit2allow -M my-dovecot#012# semodule -i my-dovecot.pp#012 is this a bad Error? When I install this local Policy i have a Problem with selinux wrong policy. sealert -a /var/log/audit/audit.log 13% donetype=AVC msg=audit(1562936830.462:61868): avc: denied { getattr } for pid=31288 comm="dovecot" path="/proc/sys/fs/suid_dumpable" dev="proc" ino=35734 scontext=system_u:system_r:dovecot_t:s0 tcontext=system_u:object_r:proc_security_t:s0 tclass=file permissive=0 **** Invalid AVC allowed in current policy *** 100% done found 0 alerts in /var/log/audit/audit.log Can any tell / help me for a correct installation? -- mit freundliche Grüßen / best regards, Günther J. Niederwimmer

1 0

[bug] success field never emited in auth_request_finished event
by Jean-Daniel Dupas 12 Jul '19

12 Jul '19

Hi, I'm playing with the new events, and encounter some issues: First the 'auth_request_finished' event is documented as having a 'successful' field, but in the code, the field is defined as 'success' (e->add_str("success", "yes")). But more important, in the function "auth_request_success_continue()" (auth/auth-request.c:288), "auth_request_log_finished(request)" is call (line 303) before updating the request status: "request->successful = TRUE" (line 312) So the log function never set the success field to "yes" as at this point request->successful is still false. Jean-Daniel

2 3

Getting login stats
by Jean-Daniel Dupas 12 Jul '19

12 Jul '19

Hello, I'm trying to get some IMAP auth stats on a Dovecot 2.3.6 instance, but whatever I declare in metric, it always show 0. What I want basically is how many IMAP auth attempts there was on the server, and optional a way to filter on the auth attempt status (successful or failed). My server uses a simple auth (with LDAP backend) and supports only "auth_mechanisms = plain login" I tried using the following metrics: -------------------------------- metric auth_request_finished { event_name = auth_request_finished } metric auth_passdb_request_finished { event_name = auth_passdb_request_finished } metric auth_userdb_request_finished { event_name = auth_userdb_request_finished } metric auth_client_request_started { event_name = auth_client_request_started } metric auth_client_userdb_lookup_started { event_name = auth_client_userdb_lookup_started } metric auth_client_passdb_lookup_started { event_name = auth_client_passdb_lookup_started } metric auth_client_cache_flush_started { event_name = auth_client_cache_flush_started } metric imap_command_finished { event_name = imap_command_finished filter { name = LOGIN } } -------------------------------- But even after many successful logins, doveadm reports 0 for all events: metric_name field count sum min max avg median stddev %95 auth_request_finished duration 0 0 0 0 0.00 0 0.00 0 auth_passdb_request_finished duration 0 0 0 0 0.00 0 0.00 0 auth_userdb_request_finished duration 0 0 0 0 0.00 0 0.00 0 auth_client_request_started duration 0 0 0 0 0.00 0 0.00 0 auth_client_userdb_lookup_started duration 0 0 0 0 0.00 0 0.00 0 auth_client_passdb_lookup_started duration 0 0 0 0 0.00 0 0.00 0 auth_client_cache_flush_started duration 0 0 0 0 0.00 0 0.00 0 imap_command_finished duration 0 0 0 0 0.00 0 0.00 0

2 2

dovecot 2.3.6 on FreeBSD 12.0-RELEASE inside a JAIL does not see dovecot/replication-notify-fifo
by Lars Schotte 12 Jul '19

12 Jul '19

Hi, I see dovecot[47939]: imap(gustik(a)gustik.eu)<48079><XVTMe3qNFpQqAchEJAsZAoGuBtbxcZmK>: Error: open(/var/run/dovecot/replication-notify-fifo) failed: No such file or directory even though, right after: Jul 12 13:54:41 madarka dovecot[47939]: aggregator: Error: Client sent invalid input when I send echo 1 > /var/run/dovecot/replication-notify-fifo and yes, of course prw-rw-rw- 1 dovecot vmail 0B Jul 12 13:57 /var/run/dovecot/replication-notify-fifo exists. FreeBSD madarka.gustik.eu 12.0-RELEASE FreeBSD 12.0-RELEASE r342152 GENERIC amd64 Dovecot running inside a jail with a version of 2.3.6 just recently compiled from ports! I ve allowed raw sockets and sysvipc for this jail but it did not help. Also making explicit imap mail plugins notify and replicator did not help. Any ideas? -- Lars Schotte Mudroňova 13 92101 Piešťany

1 0

Confused about dovecot ACL
by HTMLServices.it 12 Jul '19

12 Jul '19

Hello everyone sorry i'm not very experienced and also my english I installed a centos 7 server with ispconfig postfix dovecot 2.2.36 and roundcube, this server is only a mail archive, so my need is that ALL the mailboxes are read-only on roundcube/imap and any user must NOT delete the messages. ... so I configured dovecot's ALC following the guide "https://wiki2.dovecot.org/ACL" I think I did everything correctly and I don't get errors but entering the webmail roundcube I CAN DELETE MESSAGES ..... it seems that the acl have no effect .... added to the dovecot configuration file /etc/dovecot/dovecot.conf ______________________________________ .... plugin { acl = vfile: / etc / dovecot / dovecot-acl } ..... protocol imap { mail_plugins = $ mail_plugins imap_acl } mail_plugins = acl ..... ______________________________________ created the "global" file /etc/dovecot/dovecot-acl and inserted that the test user has only lookup and reading rights (lr): * user=test(a)test.com lr ______________________________________ my dovecot.conf ********************************* listen = *,[::] protocols = imap pop3 auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_privileged_group = vmail ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key ssl_protocols = !SSLv3 passdb { args = /etc/dovecot-sql.conf driver = sql } userdb { driver = prefetch } userdb { args = /etc/dovecot-sql.conf driver = sql } plugin { acl = vfile:/etc/dovecot/dovecot-acl quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve=/var/vmail/%d/%n/.sieve } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service imap-login { client_limit = 1000 process_limit = 500 } protocol imap { mail_plugins = $mail_plugins imap_acl mail_plugins = quota imap_quota } protocol pop3 { pop3_uidl_format = %08Xu%08Xv mail_plugins = quota } protocol lda { mail_plugins = sieve quota postmaster_address = root@localhost } protocol lmtp { postmaster_address = admin(a)htmlservices.it mail_plugins = quota sieve } mail_plugins = $mail_plugins quota mail_plugins = acl ********************************* my dovecot-acl ********************************* * user=test(a)test.com lr ********************************* "debug" ********************************* [root@archivio ~]# doveadm -Dv acl debug -u test(a)test.com INBOX Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(test(a)test.com): Debug: Added userdb setting: mail=maildir:/var/vmail/test.com/test/Maildir doveadm(test(a)test.com): Debug: Added userdb setting: plugin/quota_rule=*:storage=0B doveadm(test(a)test.com): Debug: Added userdb setting: plugin/sieve=/var/vmail/test.com/test/.sieve doveadm(test(a)test.com): Debug: Effective uid=5000, gid=5000, home=/var/vmail/test.com/test doveadm(test(a)test.com): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(test(a)test.com): Debug: maildir++: root=/var/vmail/test.com/test/Maildir, index=, indexpvt=, control=, inbox=/var/vmail/test.com/test/Maildir, alt= doveadm(test(a)test.com): Debug: acl: initializing backend with data: vfile:/etc/dovecot/dovecot-acl doveadm(test(a)test.com): Debug: acl: acl username = test(a)test.com doveadm(test(a)test.com): Debug: acl: owner = 1 doveadm(test(a)test.com): Debug: acl vfile: Global ACL file: /etc/dovecot/dovecot-acl doveadm(test(a)test.com): Info: Mailbox 'INBOX' is in namespace '' doveadm(test(a)test.com): Info: Mailbox path: /var/vmail/test.com/test/Maildir doveadm(test(a)test.com): Info: All message flags are shared across users in mailbox doveadm(test(a)test.com): Debug: Mailbox 'INBOX' matches global ACL pattern '*' doveadm(test(a)test.com): Debug: Mailbox 'INBOX' matches global ACL pattern '*' doveadm(test(a)test.com): Debug: Mailbox 'INBOX' matches global ACL pattern '*' doveadm(test(a)test.com): Debug: acl vfile: file /var/vmail/test.com/test/Maildir/dovecot-acl not found doveadm(test(a)test.com): Info: User test(a)test.com has rights: lookup read doveadm(test(a)test.com): Info: Mailbox in user's private namespace doveadm(test(a)test.com): Info: Mailbox INBOX is visible in LIST [root@archivio ~]# ********************************* if I see the line "Info: User test(a)test.com has rights: lookup read" it seems that the ACL (lookup and read) are correctly applied, but as I was saying above, entering the user test(a)test.com on the webmail, I can do everything I want to also delete the e-mails ..... these are days I try to understand but I don't understand what I'm wrong and how to solve .... thank you all in advance

2 1

Re: Best mail encryption solution for per-user
by Aki Tuomi 12 Jul '19

12 Jul '19

On 27.05.2018 21:16, mail(a)sjemm.net wrote: > May 27, 2018 8:52 AM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> On 26 May 2018 at 10:36 mail(a)sjemm.net wrote: >>> >>> May 23, 2018 10:10 AM, mail(a)sjemm.net wrote: >>> May 23, 2018 9:46 AM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> >>> On 23.05.2018 10:15, mail(a)sjemm.net wrote: >>> >>> May 23, 2018 8:31 AM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> >>> On 23.05.2018 09:13, mail(a)sjemm.net wrote: >>> May 20, 2018 8:01 PM, mail(a)sjemm.net wrote: >>> May 20, 2018 2:47 PM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> On 19 May 2018 at 16:40 mail(a)sjemm.net wrote: >>> >>> May 18, 2018 10:01 PM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> On 18 May 2018 at 21:44 mail(a)sjemm.net wrote: >>> >>> May 18, 2018 4:43 PM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> On 18 May 2018 at 17:38 mail(a)sjemm.net wrote: >>> >>> May 18, 2018 4:05 PM, "Aki Tuomi" <aki.tuomi(a)dovecot.fi> wrote: >>> On 18 May 2018 at 16:43 mail(a)sjemm.net wrote: >>> >>> Hi Tai74 and Aki, >>> I followed your conversation with interest on how to setup per user encryption in dovecot. >>> I have setup my dovecot with the following in a conf file: >>> >>> ============== >>> >>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes >>> mail_plugins = $mail_plugins mail_crypt >>> plugin { >>> >>> mail_crypt_curve = secp521r1 >>> >>> mail_crypt_save_version = 2 >>> >>> } >>> >>> ============== >>> >>> This works nice, all emails are being encrypted and every user/folder has keys. >>> But as I understood from your conversation these keys are not protected. And I want them to be >>> protected by the users password used by imap. >>> >>> Those passwords are stored in a mysql DB file. ( I used a guide from workaround [dot] org to set up >>> the DB and postfix/dovecot) >>> >>> but how would i set it so, that the users password from the DB is used to encrypt the keys? >>> >>> should i use mail_crypt_private_password = ? >>> how do i point it to the mysql db then? >>> im unsure about this >>> >>> Do you have any hints on this? >>> >>> Kind regards, >>> Zjemm >>> >>> The passwords in your MySQL database are, hopefully, not in plaintext. If you want to secure your >>> user's keys using user's login password, you must have a TOOL that manages this. >>> >>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login >>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be >>> a good option. >>> >>> You can change the key password using 'doveadm mailbox cryptokey', this needs to be done every time >>> user changes his password. >>> >>> Also note that if you go down this road, and the user forgets his password, you will not be able to >>> recover the emails without backup copy of the private key. >>> >>> Aki >>> >>> Hi Aki >>> >>> I used the following command: >>> dovecot pw -s SHA256-CRYPT >>> >>> the output on the chosen password looks like: {SHA256-CRYPT}$5$Rokc06a7In4SF3bO$OQpGQWqg........ >>> >>> This output is used to store in the password fields in the database. So no plain text passwords no >>> :) >>> >>> You can use mail_crypt_private_password = %w in (mysql) passdb fields to provide the user's login >>> password as private password. >>> >>> can you explain this a bit more for me? >>> >>> for now i have in the 10-auth.conf file the following: >>> ============== >>> passdb { >>> driver = sql >>> >>> # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext >>> args = /etc/dovecot/dovecot-sql.conf.ext >>> } >>> >>> and: >>> >>> userdb { >>> driver = static >>> args = uid=vmail gid=vmail home=/var/vmail/%d/%n >>> } >>> ============== >>> >>> then i have in dovecot-sql.conf.ext >>> ============== >>> driver = mysql >>> connect = host=x.x.x.x dbname=mailserver user=mailuser password=mailpasswordexample >>> default_pass_scheme = SHA256-CRYPT >>> password_query = SELECT email as user, password FROM virtual_users WHERE email='%u'; >>> ============== >>> Where do i need to set : mail_crypt_private_password = %w ? >>> >>> password as private password. You might want to run it thru some hash, so %{sha1:password} might be >>> a good option. >>> >>> the passwords are allready hashed in the DB using: dovecot pw -s SHA256-CRYPT to genereate the has. >>> so this step isnt nesesary anymore am i right? >>> >>> Thank you for your quick response, very helpfull >>> >>> Zjemm >>> >>> You misunderstood a bit. The idea is to use the *plaintext* password as the password for the >>> private key. Otherwise anyone could just decrypt it by looking at your database where the hashed >>> password is.. >>> >>> So: >>> >>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM >>> virtual_users WHERE email='%u' >>> >>> Aki >>> >>> Hi Aki, >>> >>> Thank you very much for your help, i realy appreciate that. >>> >>> Ok so if i understand it correctly i'll have to use: >>> >>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM >>> virtual_users WHERE email='%u' >>> >>> in my dovecot-sql.conf.ext file >>> >>> This query selects the user, the password, and %w >>> >>> if i run a little query myself: >>> MariaDB [mailserver]> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password >>> FROM virtual_users; >>> +------------------+----------------------------------+------------------------------------+ >>> | user | password | userdb_mail_crypt_private_password | >>> +------------------+----------------------------------+------------------------------------+ >>> | john(a)example.org | {SHA256-CRYPT}$5$M/GWzmtjsLroRWI | %w | >>> +------------------+----------------------------------+------------------------------------+ >>> >>> %w is a dovecot variable, and stands for the plaintext password, but the password is not stored as >>> plaintext in the DB, %w get filled with the actual plaintext password by dovecot upon the user that >>> is typing in the password when authenticating. >>> >>> is this correct? >>> >>> yes. >>> >>> so then i have the username the hashed password en the plaintext password as a result of the query. >>> >>> yes >>> >>> now userdb_mail_crypt_private_password = the plaintext password >>> do i need to reference it somewhere? or is userdb_mail_crypt_private_password autmatically used by >>> the dovecot mail_crypt plugin to encrypt the keys? or should it be mail_crypt_private_password? >>> >>> It gets injected into the mail process as 'mail_crypt_private_password', as if it was set in plugin >>> {} section. >>> >>> if i have this setup working i'm going to write a blog post on this topic to share this knowledge >>> >>> Thanks again and have a great weekend. >>> >>> Zjemm >>> >>> Aki >>> >>> Hi Aki, >>> >>> Cool i'm testing it right now. >>> I have set up a new mailserver (life is great with lxc containers :) ) >>> >>> postfix and dovecot are working like normal >>> >>> next i enable mail_crypt >>> >>> i did create a file: /etc/dovecot/conf.d/10-mailcrypt.conf >>> ========================== >>> mail_attribute_dict = file:%h/Maildir/dovecot-attributes >>> >>> mail_plugins = $mail_plugins mail_crypt >>> >>> plugin { >>> mail_crypt_curve = secp521r1 >>> mail_crypt_save_version = 2 >>> } >>> ========================== >>> >>> and then i changed the file: /etc/dovecot/dovecot-sql.conf.ext >>> >>> so the query is now the new query: >>> password_query = SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM >>> virtual_users WHERE email='%u'; >>> >>> then i restarted dovecot and postfix and send a test email to the one and only testuser that is in >>> there. >>> >>> when i open the mailbox with the tool mutt, i can see the new email, and when openening the email >>> the mutt client drops the connection. >>> >>> in the log i can see: >>> >>> May 19 13:34:48 mailserver1.example.local dovecot[600]: imap-login: Login: user=<john(a)example.org>, >>> method=PLAIN, rip=::1, lip=::1, mpid=713, TLS, session=<E3PnIY9sNM4AAAAAAAAAAAAAAAAAAAAB> >>> May 19 13:34:49 mailserver1.example.local dovecot[600]: imap(john(a)example.org): Error: read() >>> failed: >>> read(/var/vmail/example.org/john/Maildir/cur/1526736378.M161472P641.mailser…, >>> =571:2,) failed: Private key not available: Cannot decrypt key >>> bfc5bb25b1bf64290eea6dc14b516c6a0a25b64551b6e4f0f8677ba7274887cb: error:03070068:bignum >>> routines:BN_mpi2bn:encoding error (FETCH BODY[] for mailbox INBOX UID 8) >>> >>> i think i missed a step, but witch one? >>> >>> the userpassword hasnt been changed (that would be the next step in the testing process) >>> >>> should i've use doveadm first to encrypt the key with that userpassword? i thought it would do that >>> on the fly, because the initial keys where only just created when enabling the mail_crypt plugin >>> >>> please let me know your thougts >>> Zjemm >>> I noticed you replied directly to me, and not to the list, too... fixed that for you. >>> >>> mail_crypt_private_password is used when key is created, but if you have created it before using >>> password, you'll need to encrypt it before turning the setting on. >>> >>> Aki >>> Hi, Ok i'm sorry about that, thank you for fixing that. i'm a bit new to mailing lists :) >>> >>> are you a dovecot devoloper? or a enthusiastic user? >>> >>> anyway, i managed to get it working now with all your help :) >>> >>> to start fresh i did a: >>> rm -rf /var/vmail/example.org >>> >>> so all email is gone, and i assume all dovecot keys are gone that way. >>> >>> Then i tried to set a password first: >>> >>> root@mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john(a)example.org -n summersun >>> result: Changed password for 0 key(s) >>> >>> (summersun = the password for this user) >>> >>> but then it says there are 0 keys, so then i send the test user an email and try to open the >>> mailbox, i'll get: >>> >>> imap(john(a)example.org): Error: read() failed: >>> read(/var/vmail/example.org/john/Maildir/cur/1526827992.M149712P883.mailser…, >>> =561:2,) failed: Private key not available: Cannot decrypt key >>> 71849013a70b0c631c06112077e7c2fe39b0b2737b4933b219793841209d4e7f: error:03070068:bignum >>> routines:BN_mpi2bn:encoding error >>> >>> but hey, now the key's are created, so i tried to set the password again: >>> >>> root@mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john(a)example.org -n summersun >>> result: dcrypt_key_load_private(787701bd677dd69a26842547a3926cbaa625b0b5a91751f06678c3e9708343d9) >>> failed: password missing >>> ========= >>> >>> Lets start over again, i did a new: >>> rm -rf /var/vmail/example.org >>> >>> i disabled the mail_crypt_private_password in the sql query and send an email to the test user. >>> now again new keys are generated. >>> >>> then i did set the password via doveadm: >>> >>> root@mailserver1:/var/vmail# doveadm mailbox cryptokey password -u john(a)example.org -n summersun >>> result: Changed password for 1 key(s) >>> >>> i enabled the mail_crypt_private_password again in the sql query part and now it works :) >>> >>> YESSS!!! >>> >>> if i now change the password for the user, i cannot read the email anymore. that is what should >>> happen. >>> so now i changed the password for this user also with doveadm: >>> - doveadm mailbox cryptokey password -u john(a)example.org -n qwerty -o summersun >>> >>> and now i'm able to read the emails again. so it all works now as expected. >>> >>> i have now only one thing remaining. >>> >>> when i want to add a new user, i have to somehow: >>> - create the new user in the mysql DB >>> - start with dovecot not using mail_crypt_private_password (so disabled) >>> - send an initial email to let the users mailbox be created and generate the keys >>> - set the users password with doveadm >>> - restart dovecot again with mail_crypt_private_password (so enabled) >>> >>> but this would be disruptive to existing users, and i would have to script the whole thing as a >>> "add-new-user" script to make it usable i guess. >>> >>> do you know of a more easy way of acomplishing this? >>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part? >>> >>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that >>> column with %w for users that allready exist, and empty for new users. >>> when being empty, it might not set the mail_crypt_private_password for new users >>> >>> or is this the wrong way of thinking? >>> >>> let me know your thoughts on this. >>> >>> And last but not least, i really want to thank you for all your help. It is a really nice >>> experience. >>> Bless you! >>> >>> Zjemm >>> Hi Aki, >>> >>> You did fix the posting on the mail archive, i can see our conversation on there, all but the last >>> email i send to you. >>> Maybe you did not received the last email? >>> >>> would you be willing to take a look at my last question in the email See below: >>> Thank you very much. >>> >>> ================================ >>> i have now only one thing remaining. >>> >>> when i want to add a new user, i have to somehow: >>> - create the new user in the mysql DB >>> - start with dovecot not using mail_crypt_private_password (so disabled) >>> - send an initial email to let the users mailbox be created and generate the keys >>> - set the users password with doveadm >>> - restart dovecot again with mail_crypt_private_password (so enabled) >>> >>> but this would be disruptive to existing users, and i would have to script the whole thing as a >>> "add-new-user" script to make it usable i guess. >>> >>> do you know of a more easy way of acomplishing this? >>> could i use doveadm for all steps, and get around the mail_crypt_private_password disable part? >>> >>> or could i add a column in the mysql db called userdb_mail_crypt_private_password and fill that >>> column with %w for users that allready exist, and empty for new users. >>> when being empty, it might not set the mail_crypt_private_password for new users >>> >>> or is this the wrong way of thinking? >>> >>> let me know your thoughts on this. >>> >>> And last but not least, i really want to thank you for all your help. It is a really nice >>> experience. >>> Bless you! >>> >>> Zjemm >>> >>> If you have mail_crypt_private_password set *when* keys are generated, >>> dovecot should use that password to encrypt the user key. >>> >>> Aki >>> Hi Aki, >>> >>> well, if there is a new email user setup in the mysql DB, and no email has been send to that new >>> user. at that point there are no keys. >>> >>> So trying to set a password on those key's wont work. see: >>> root@mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2(a)example.org -n >>> qwerty >>> result: Changed password for 0 key(s) >>> >>> the keys need to be generated first. And that only happens when sending a mail to that new user. >>> Also, at the point of creating the keys, the mail_crypt_private_password should be disabled >>> otherwise the keys are generated and also encrypted with no user password, and then im unable to >>> set a new pasword for that key >>> >>> How can i set the password before the keys are generated? >>> or can i manually generate key's and then set the password? >>> >>> so at this point i have to do the following: >>> >>> You can use doveadm mailbox cryptokey generate -u user -U >>> >>> to generate a user key. >>> >>> Aki >>> >>> - create the new user in the mysql DB >>> - start with dovecot not using mail_crypt_private_password (so disabled) >>> - send an initial email to let the users mailbox be created and generate the keys >>> - set the users password with doveadm >>> - restart dovecot again with mail_crypt_private_password (so enabled) >>> can i somehow enable the mail_crypt_private_password per user? >>> because this mysql query always returns: %w >>> and thus allways sets: userdb_mail_crypt_private_password >>> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users; >>> >>> Hi Aki, >>> >>> You are correct >>> >>> doveadm mailbox cryptokey generate -u user -U generates the keys, but i have to first remove: >>> '%w' AS userdb_mail_crypt_private_password >>> from dovecot configuration and restart dovecot. >>> >>> if i don't do that, the keys are generated, but also somehow encrypted using: no >>> password/empty/something else (i don't know) >>> >>> here you can see that happening: >>> >>> root@mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey generate -u john2(a)example.org -U >>> Folder Public ID >>> ✓ <userkey> 53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4 >>> root@mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2(a)example.org -n >>> qwerty >>> result: dcrypt_key_load_private(53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4) >>> failed: password missing >>> >>> it expects an old password wich is never set in my opinion. >>> >>> so the last part of this setup would be to figure out a way to disable '%w' AS >>> userdb_mail_crypt_private_password for new users without manually removing that part from the query >>> and restarting dovecot. As that would mean connection interruption while adding new users >>> >>> if we could add new users to the DB and somehow not give '%w' AS userdb_mail_crypt_private_password >>> for that new user in the query, we can add the password afterward, and then enable '%w' AS >>> userdb_mail_crypt_private_password in the query for this user >>> >>> could we add a column in the user db lets say called: encrypted (yes/no) >>> >>> and then change the existing query: SELECT email as user, password, '%w' AS >>> userdb_mail_crypt_private_password FROM virtual_users; >>> >>> to firrt check if encryption is set to 'yes', if so retrun: >>> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users; >>> >>> if not return: >>> SELECT email as user, password FROM virtual_users; >>> >>> but then in one mysql query so dovecot can work with it >>> >>> ? >>> >>> Hi Aki, >>> >>> would you be willing to respond to my last question i had send to you last week? >>> its this one: >>> ================================== >>> doveadm mailbox cryptokey generate -u user -U generates the keys, but i have to first remove: >>> '%w' AS userdb_mail_crypt_private_password >>> from dovecot configuration and restart dovecot. >>> >>> if i don't do that, the keys are generated, but also somehow encrypted using: no >>> password/empty/something else (i don't know) >>> >>> here you can see that happening: >>> >>> root@mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey generate -u john2(a)example.org -U >>> Folder Public ID >>> ✓ <userkey> 53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4 >>> root@mailserver1:/var/vmail/example.org# doveadm mailbox cryptokey password -u john2(a)example.org -n >>> qwerty >>> result: dcrypt_key_load_private(53004acf9886f887ca081c83d7392a5aa3fac4b5a20ce013db91a341fb4644c4) >>> failed: password missing >>> >>> it expects an old password wich is never set in my opinion. >>> >>> could this be a bug? or am i doing it wrong? if needed i could send in a bug report? >>> >>> or might we need to enable the userdb_mail_crypt_private_password "per user" ? >>> >>> so the last part of this setup would be to figure out a way to disable '%w' AS >>> userdb_mail_crypt_private_password for new users without manually removing that part from the query >>> and restarting dovecot. As that would mean connection interruption while adding new users >>> >>> if we could add new users to the DB and somehow not give '%w' AS userdb_mail_crypt_private_password >>> for that new user in the query, we can add the password afterward, and then enable '%w' AS >>> userdb_mail_crypt_private_password in the query for this user >>> >>> could we add a column in the user db lets say called: encrypted (yes/no) >>> >>> and then change the existing query: SELECT email as user, password, '%w' AS >>> userdb_mail_crypt_private_password FROM virtual_users; >>> >>> to firrt check if encryption is set to 'yes', if so retrun: >>> SELECT email as user, password, '%w' AS userdb_mail_crypt_private_password FROM virtual_users; >>> >>> if not return: >>> SELECT email as user, password FROM virtual_users; >>> >>> but then in one mysql query so dovecot can work with it >>> >>> Please let me know your thougts, after this last thing im done :) >>> also let me know if i can support this project in some way, im willing to contribute >>> >>> Zjemm >> You can select it as userdb_mail_crypt_save_version=0 to disable encryption (it won't disable >> decryption). You can use value '2' for enabling encryption. DO NOT USE 1. >> >> Also I tried the SQL based thing, and if you had EMPTY mail_crypt_private_password, try these to >> change the password. If the one that works is the second one, let me know =) >> >> doveadm mailbox cryptokey password -u testuser -nnewpass -o "" >> >> doveadm mailbox cryptokey password -u testuser -nnewpass -o "yes" >> >> Aki > Hi Aki, > > Thanks for replying > > So if i understand you correctly i could enable encryption per user by using: userdb_mail_crypt_save_version= in the database as an extra column. the column would be called "userdb_mail_crypt_save_version" and the value is "0" or "2" > > if a user has: userdb_mail_crypt_save_version=0, encryption is disabled for that user > if a user has: userdb_mail_crypt_save_version=2, encryption is enabled for that user > > would this also override the setting: mail_crypt_save_version = 2 > in the mailcryp.conf file where that is a global setting? or should i delete it there and only use userdb_mail_crypt_save_version= from the database? > > for the password set command i hve testest some things: > > new user with no keys yet: > doveadm -o plugin/mail_crypt_private_password=password mailbox cryptokey generate -u john2(a)example.org -U > > i can set a password with: > new password: > doveadm mailbox cryptokey password -u john2(a)example.org -n newpass -o password > > > if i do not use new user doveadm: (doveadm -o plugin/mail_crypt_private_password=password mailbox cryptokey generate -u john2(a)example.org -U) > then use this: > doveadm mailbox cryptokey password -u john2(a)example.org -n newpass -o "yes" > > > so yes it is the second one > > Kind regards, > > Zjemm It's unfortunate problem with 2.2 that it defaults empty userdb variables into yes. This has been remedied in 2.3. Unfortunately there is no clean fix to this, they all resort to various levels of trickery. Variables set in passdb/userdb overwrite the ones in plugin {} section. Aki

2 2

Connection statistics per user and IP
by Lefteris Tsintjelis 11 Jul '19

11 Jul '19

Is it possible to enable statistics and keep track of the maximum number of connections per user and also per IP? Lefteris

3 2

Frequent Out of Memory for service(config)
by Root Kev 11 Jul '19

11 Jul '19

Hello Group, We have dovecot deployed as solely a Pop3 service that is used by our applications to pass mail from one application to another internally. We have roughly 4 applications that connect to the Pop3 service every 2 seconds, to check for new messages and pop them for processing if they are present. Depending on the site, we have between 1024-2048MB of memory set for default_vsz_limit. In all systems we see the Out of memory alert several times a day. We previously did not see this at all when running on CentOS6, with less memory. We have tried increasing the memory to the vsz_limit up to 2gb without success. We are running on CentOS 7 servers, running dovecot 2.3.6 (7eab80676) (from the dovecot repo). Can anyone advise any other settings that could be modified in order to correct these out of memory issues? # dovecot -n # 2.3.6 (7eab80676): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: ****** #<modified for security> auth_cache_size = 10 M auth_verbose = yes default_vsz_limit = 1 G instance_name = Pop3 Mail Service listen = 10.*.*.* #<modified for security> log_path = /var/log/dovecot.log login_greeting = Pop3 Mail Service login_trusted_networks = 10.*.*.* 10.*.*.* 10.*.*.* 10.*.*.* 10.*.*.* #<modified for security> mail_location = maildir:~/Maildir namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = cache_key=#hidden_use-P_to_show# driver = pam } protocols = pop3 ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } verbose_ssl = yes May 10 06:44:05 config: Fatal: pool_system_malloc(8192): Out of memory May 10 06:44:05 config: Fatal: master: service(config): child 27887 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 10 14:56:31 config: Fatal: pool_system_malloc(8192): Out of memory May 10 14:56:31 config: Fatal: master: service(config): child 2495 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 10 23:09:02 config: Fatal: pool_system_malloc(8192): Out of memory May 10 23:09:02 config: Fatal: master: service(config): child 10581 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 11 10:18:43 config: Fatal: pool_system_malloc(8192): Out of memory May 11 10:18:43 config: Fatal: master: service(config): child 30511 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 11 18:30:59 config: Fatal: pool_system_malloc(1608): Out of memory May 11 18:30:59 config: Fatal: master: service(config): child 12134 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 12 02:57:28 config: Fatal: pool_system_malloc(8192): Out of memory May 12 02:57:28 config: Fatal: master: service(config): child 24940 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 12 11:07:34 config: Fatal: pool_system_malloc(8192): Out of memory May 12 11:07:35 config: Fatal: master: service(config): child 8203 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 12 19:17:52 config: Fatal: pool_system_malloc(8192): Out of memory May 12 19:17:52 config: Fatal: master: service(config): child 14623 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 13 03:26:34 config: Fatal: pool_system_malloc(8192): Out of memory May 13 03:26:35 config: Fatal: master: service(config): child 21479 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 13 11:35:42 config: Fatal: pool_system_malloc(8192): Out of memory May 13 11:35:43 config: Fatal: master: service(config): child 26900 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) May 13 19:34:26 config: Fatal: pool_system_malloc(8192): Out of memory May 13 19:34:26 config: Fatal: master: service(config): child 32187 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump). Thanks, Kevin

4 6

Re: Segfault of doveconf
by Claudio Prono 11 Jul '19

11 Jul '19

On 9.7.2019 13.40, Aki Tuomi via dovecot wrote: On 9.7.2019 12.54, Claudio Prono wrote: On 4.7.2019 19.00, Claudio Prono via dovecot wrote: Sorry to bother again, but the problem persist. I have checked all the configurations of dovecot, and seems all ok, but randomly the segfault pops up again, without no apparently reason. My packages are: ii dovecot-core 1:2.2.22-1ubuntu2.10 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.22-1ubuntu2.10 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-managesieved 1:2.2.22-1ubuntu2.10 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.22-1ubuntu2.10 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.22-1ubuntu2.10 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.22-1ubuntu2.10 amd64 secure POP3/IMAP server - Sieve filters support Can be a temporarly solution to uninstall dovecot-managesieved ? Cordially, Claudio. On 5.7.2019 09:59 Aki via dovecot wrote: You should install dovecot-dbg, and follow https://www.dovecot.org/bugreport-mail, then we could figure out why it's crashing in the first place. Aki I have followed the instructions to debug the problem, and I have uploaded the crash log at this location: https://justpaste.it/5wlot Yesterday the system have got some I/O Errors on the hard disk (virtual system hosted from Seeweb). The problems on the disk now are solved by the provider, and the crash reported is before that. So, now my question is: the doveconf crash can be linked to the hardware problem ? Or something else? Thanks for your help. Cordially, Claudio Prono. I tried unpacking the crash file you pastebin, can you please send it as attachment to me? Aki It seems you are at least missing postmaster_address in your config. I was not able to actually make dovecot crash when I tested, so can you try adding this setting and see if it helps? Aki On 9.7.2019 09:59 Claudio Prono via dovecot wrote: I have added it to the configuration, and restarted dovecot. Now I wait and see if this error again. Thank you for your time. Cheers, Claudio. ---- Just a rapid update, the problem of the segmentation fault of doveconf was not occourred again in the past two days. I think the problem was somehow related to the hardware failure on the virtual disk where the system is hosted. Now, the hardware problem is resolved from the virtual system maintainer, and doveconf has stopped to crash. Anyway, thanks for the support. Claudio.

1 0

Dovecot 2.3.6 on Solaris10: build issues, segfaults
by Joseph Tam 11 Jul '19

11 Jul '19

Hopefully, there is some fix for issue 3 which is beyond my skill to fix. Issue 1) Need recent gcc version Building Dovecot versions <=2.2.x using gcc 3.4.4 worked, but this gcc version fails to build 2.3.x properly: symptoms include compile failures and executable crashes that depended on the amount of optimization used, which is usually a sign of compiler bugs. (It could also be issue 3 in disguise.) Either way, I updated to gcc 9.1.0. Issue 2) Cannot build with --enable-hardening Using gcc 9.1.0, "configure" step fails because fd passing was broken, but the real problem was a compilation failure when "--enable-hardening" is used. Demonstration: # echo 'int main(){char a[1]; strcpy(a,a);} ' | gcc -w -fstack-protector-strong -x c - Undefined first referenced symbol in file __stack_chk_guard /var/tmp//cc12L9zV.o (symbol scope specifies local binding) ld: fatal: symbol referencing errors. No output written to a.out collect2: error: ld returned 1 exit status I'm not sure if this is a Solaris10 fumble, but configuring "--disable-hardening" removes the "-fstack-protector-strong" compiler option, which resolves this issue. Issue 3) dovecot/doveconf segfaults on startup It crashes here while processing dovecot.conf, as does "doveconf" (settings-parser.c:1519 in setting_copy()) *dest_size = *src_size; It appears *src_size is not an 8-byte address aligned (0x5597c). It inherits this value from the calling routine as the sum of "set" (8-byte aligned) + "def->offset"=20 => misaligned address. (settings-parser.c:1597 in settings_dup_full()) src = CONST_PTR_OFFSET(set, def->offset); (gdb) p set $2 = (const void *) 0x55968 (gdb) p *def $3 = {type = SET_SIZE, key = 0x2d548 "submission_max_mail_size", offset = 20, list_info = 0x0} (gdb) bt full #0 0xff190690 in setting_copy (type=SET_SIZE, src=0x5798c, dest=0xf7ed4, pool=0xf29a0, keep_values=false) at settings-parser.c:1519 src_size = 0x5798c dest_size = 0xf7ed4 __func__ = "setting_copy" #1 0xff190bc4 in settings_dup_full (info=0x2d9ac <submission_setting_parser_info>, set=0x57978, pool=0xf29a0, keep_values=false) at settings-parser.c:1600 def = 0x2d7d4 <submission_setting_defines+80> src = 0x5798c dest_set = 0xf7ec0 dest = 0xf7ed4 children = 0x2c i = 1105688 count = 4279837964 #2 0xff192724 in settings_parser_dup (old_ctx=0x9ec08, new_pool=0xf29a0) at settings-parser.c:1946 new_ctx = 0x10def0 iter = 0x0 links = {_table = 0xa23c0, _key = 0xa23c0, _keyp = 0xa23c0, _const_key = 0xa23c0, _value = 0xa23c0, _valuep = 0xa23c0} new_link = 0x10b488 value = 0x0 key = 0x0 i = 0 parser_pool = 0x10ded8 keep_values = false __func__ = "settings_parser_dup" #3 0x0001fc0c in config_filter_parsers_get (ctx=0x657a8, pool=0xf29a0, modules=0x0, filter=0x583e8, parsers_r=0xffbff9c4, output_r=0xffbff9bc, error_r=0xffbffa44) at config-filter.c:372 src = 0xf29d8 dest = 0xf2a48 error = 0x0 error_p = 0x52cfc i = 26 count = 33 __func__ = "config_filter_parsers_get" #4 0x00021be4 in config_all_parsers_check (ctx=0xffbffac8, new_filter=0x657a8, error_r=0xffbffa44) at config-parser.c:441 parsers = 0x58180 tmp_parsers = 0x4 output = {specific_services = 0xf2a20, service_uses_local = false, service_uses_remote = false, used_local = false, used_remote = false, permission_denied = false} i = 0 count = 3 ssl_set = 0x4 <error: Cannot access memory at address 0x4> global_ssl_set = 0x32188 "" tmp_pool = 0xf29a0 ssl_warned = false ret = 0 __func__ = "config_all_parsers_check" #5 0x00022ec8 in config_parse_finish (ctx=0xffbffac8, error_r=0xffbffb98) at config-parser.c:747 new_filter = 0x657a8 error = 0x33c22 "/user=%s" ret = 0 #6 0x00024230 in config_parse_file (path=0x50468 "/local/dovecot/etc/dovecot/dovecot.conf", expand_values=false, modules=0x0, error_r=0xffbffb98) at config-parser.c:1064 root = {prev = 0x0, input = 0x0, path = 0x50468 "/local/dovecot/etc/dovecot/dovecot.conf", linenum = 400} ctx = {pool = 0x52b58, path = 0x50468 "/local/dovecot/etc/dovecot/dovecot.conf", , all_parsers = {arr = {buffer = 0x58160, element_size = 4}, v = 0x58160, v_modifiable = 0x58160}, root_parsers = 0x52b70, cur_section = 0x58388, cur_input = 0x0, str = 0x58458, pathlen = 0, section_counter = 6, error = 0x0, old = 0x58580, seen_settings = {_table = 0x0, _key = 0x0, _keyp = 0x0, _const_key = 0x0, _value = 0x0, _valuep = 0x0}, filter = 0x0, expand_values = false, hide_errors = false} i = 33 count = 33 key = 0x0 value = 0x0 full_line = 0x0 type = CONFIG_LINE_TYPE_SKIP line = 0x0 fd = -1 ret = 0 handled = false #7 0x000188dc in main (argc=1, argv=0xffbffc94) at doveconf.c:979 master_service_flags = (MASTER_SERVICE_FLAG_STANDALONE | MASTER_SERVICE_FLAG_NO_INIT_DATASTACK_FRAME | MASTER_SERVICE_FLAG_DONT_SEND_STATS) scope = CONFIG_DUMP_SCOPE_ALL orig_config_path = 0x46300 "/local/dovecot/etc/dovecot/dovecot.conf" config_path = 0x50468 "/local/dovecot/etc/dovecot/dovecot.conf" module = 0x1c00 <error: Cannot access memory at address 0x1c00> module_names = {arr = {buffer = 0x46338, element_size = 4}, v = 0x46338, v_modifiable = 0x46338} filter = {service = 0x0, local_name = 0x0, local_host = 0x0, remote_host = 0x0, local_net = {family = 0, u = {ip6 = {_S6_un = {_S6_u8 = '\000' <repeats 15 times>, _S6_u32 = {0, 0, 0, 0}, __S6_align = 0}}, ip4 = {S_un = {S_un_b = { s_b1 = 0 '\000', s_b2 = 0 '\000', s_b3 = 0 '\000', s_b4 = 0 '\000'}, S_un_w = { s_w1 = 0, s_w2 = 0}, S_addr = 0}}}}, remote_net = {family = 0, u = {ip6 = { _S6_un = {_S6_u8 = '\000' <repeats 15 times>, _S6_u32 = {0, 0, 0, 0}, __S6_align = 0}}, ip4 = {S_un = {S_un_b = {s_b1 = 0 '\000', s_b2 = 0 '\000', s_b3 = 0 '\000', s_b4 = 0 '\000'}, S_un_w = {s_w1 = 0, s_w2 = 0}, S_addr = 0}}}}, local_bits = 0, remote_bits = 0} wanted_modules = 0x0 error = 0xfef47940 <_uberdata> "" exec_args = 0x0 setting_name_filters = 0x0 i = 5 c = -1 ret = -4195172 ret2 = 4 config_path_specified = false expand_vars = false hide_key = false parse_full_config = false simple_output = false dump_defaults = false host_verify = false print_plugin_banner = true hide_passwords = true Joseph Tam <jtam.home(a)gmail.com>

4 6

Dovecot behind Load Balancer
by Paolo Daniele 10 Jul '19

10 Jul '19

Hi, i've a question for you. I've two dovecot imap/pop server behind a zen load balancer. Load balancing is made by lx4nat so the public ip address of my load balancer contact directly the dovecot servers. Since few months i've a message from thunderbird that i've reached the imap limit login for ip. I've triend to increase the max user ip parameter but sometimes i've the same problem. It's a strange things that actually i'm able to mitigate by reduce the number of cached connections in Thunderbird but it's not normal. What do you think about that? Maybe there's some tuning that you can suggest. Thank you, Paolo

5 10

solr
by Maciej Milaszewski IQ PL 10 Jul '19

10 Jul '19

Hi I have set up SOLR in accordance with documentation and it runs well. I use solr like: ....... fts = solr fts_solr = debug url=http://IP:8983/solr/ (solr in external machine) ...... Is replication of this system really essential ?? Due to my tests, rollback of the solr server on external machine lasts less than minute and it is nearly non-visible from client side. On the other hand solr replication is quite complicated process and rollback or master-slave switch in this case is non-trivial task, that may have result in whole dataset inconsistency. Do you have any experience in such cases ? Maby load-balance in HAProxy colud do the thing ? Something like: ......... server search1 192.168.1.1:8983 check port 8983 inter 20s fastinter 2 server search2 192.168.1.2:8983 backup ......... Best Regards

2 1

Some questions
by Jérôme Bardot 10 Jul '19

10 Jul '19

Hello, This is my first email here. I want to understand well how dovecot is integrate with ldap in a postfix/dovecot/ldap setup. I use a debian server. More specifically what dovecot need in ldap to work. I saw we can use several "mode" related to virtual domain, etc. For "start" i only need one domain with several address. I currently use fusiondirectory for manage my ldap users. i guess i can use that schema to auto create users email (name.firstname(a)domain.tld for ie) ? I also want to setup some aliases and share directory based on ldap group/role can i do it ? An other question is can we have two domain name for imap.domain.tld && smtp.domain.tld ? IF some questions are out of topic let me know. thx jerome

4 3

Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS!
by Arnold Opio Oree 10 Jul '19

10 Jul '19

Hello Timo, I have scheduled some time to carry through the test requested by Aki. I will add responding to you as part of that process. Not sure if I have said or done anything to offend you, as your tone appears quite abrasive, even where you do not have an actual response to an observation. It is my view that it is a privilege to provide an open source platform, and therefore a responsibility wherever feasible not only to properly maintain and support it, but also to properly document it; both of which from my experience and findings to date, Dovecot have failed to do. Kind regards, Arnold Opio Oree Chief Executive Officer Parallax Digital Technologies arnoldoree(a)parallaxdt.com http://www.parallaxdt.com tel : +44 (0) 333 577 8587 fax : +44 (0) 20 8711 2477 Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. 08836288 The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other persons. -----Original Message----- From: Timo Sirainen via dovecot <dovecot(a)dovecot.org> Reply-To: Timo Sirainen <timo(a)sirainen.com> To: Arnold Opio Oree <arnold.oree(a)parallaxict.com> Cc: Dovecot Mailing List <dovecot(a)dovecot.org> Subject: Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS! Date: Wed, 10 Jul 2019 11:09:57 +0300 On 7 Jul 2019, at 18.12, Arnold Opio Oree via dovecot < dovecot(a)dovecot.org> wrote: > > Dovecot Team, > > > > I'd like to report a number of bugs, that are to my view all > > critical. It would help to get your doveconf -n, example command lines causing the problems and the error messages it outputs or what the wrong behavior looks like in filesystem. It's now rather difficult to guess what exactly you tried and what happened. Also what kind of output does readpst make? I'm not sure why you're using DIRNAMEs here. > > doveadm-sync -1/general > > > > 1) If DIRNAMEs are not different between command line and > > mail_location doveadm sync will fail, saying that the source and > > destination directories are the same This sounds very strange. I'm not sure what exactly you did, and I couldn't reproduce with a small test. > > 2) The -n / -N flags do not work, and a sync will fail strangely if > > location is specified in the namespace definition Again, sounds strange. > > 3) Adds mbox to path name under mailbox directory (where syncing > from > > an mbox source) Probably with different parameters you could avoid it. > > 4) Not having the mailboxes at source named the same as those at > > destination causes errors and partial sync > > > > 5) Not having the target mailboxes formatted to receive the sync > > (/<mailboxroot>/DIRNAME/) will cause sync errors. I don't understand these. Target mailboxes are supposed to be empty initially, and after the initial sync they should be in the expected format. Why would they be different? > > doveadm-sync > > > > 1) With large synchronizations UIDs are corrupted where multiple > > syncs are executed and the program can no longer synchronize What exactly is the error message? > > dovecot > > > > 1) Panics and fails to expand ~ to user home: observed cases are > > where multiple namespaces are being used Panic message and more details would also help. > > With regards to the last error that I requested help on i.e. > > \Noselect. This has been resolved more-or-less by the workarounds > > that I have implemented for the bugs reported above. > > > > I have seen a number of threads whilst researching the \Noselect > > issue where people have been very confused. My finding was that > > \Noselect is a function of the IMAP specification server-side > > implementation RFC3501 ( > > https://tools.ietf.org/html/rfc3501#section-6.3.6) And for me the > > server was returning directories with \Noselect because the > mailboxes > > were malformed on account of dovadm-sync errors. In order to fix > this > > I formed a bash command to transverse the mailbox hierarchy and > > create the missing folders critical to the sdbox format, namely > > DIRNAME. Nowadays we have also an option to disable creation of \Noselect folders, because they confuse people. Using mail_location = ...:NO- NOSELECT - It won't affect existing folders immediately though. -----Original Message----- From: Arnold Opio Oree <arnoldoree(a)parallaxict.com> Reply-To: arnoldoree(a)parallaxict.com To: Aki Tuomi <aki.tuomi(a)open-xchange.com>, Dovecot Mailing List < dovecot(a)dovecot.org> Cc: debian-release(a)lists.debian.org Subject: Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS! Date: Mon, 08 Jul 2019 13:21:51 +0100 Hello Aki, Thanks for looking into these. I will as requested attempt the relevant procedures under Dovecot 2.3.6. To make the test fair, I will need to fork the relevant production groupware stack (which is now stable and in operation, with our enterprise (email) data successfully migrated from Microsoft Exchange) to a new staging server; given that the current staging server is now of a materially different configuration to the production server (where the most controlled observations of these bugs were made). Kindly give me some time, as I have an urgent internal openstack deployment project to kick-off, that has been delayed by an overrun of this internal groupware stack deployment project. Best regards, Arnold Opio Oree Chief Executive Officer Parallax Digital Technologies arnoldoree(a)parallaxdt.com http://www.parallaxdt.com tel : +44 (0) 333 577 8587 fax : +44 (0) 20 8711 2477 Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. 08836288 The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other persons. -----Original Message----- From: Aki Tuomi via dovecot < dovecot(a)dovecot.org > Reply-To: Aki Tuomi < aki.tuomi(a)open-xchange.com > To: arnoldoree(a)parallaxict.com , Arnold Opio Oree < arnold.oree(a)parallaxict.com >, Arnold Opio Oree via dovecot < dovecot(a)dovecot.org > Cc: debian-release(a)lists.debian.org Subject: Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS! Date: Mon, 8 Jul 2019 08:48:08 +0300 (EEST) Hi! Thank you for reporting these. We will look into them. In the mean time, can you see if any of these are fixed in 2.3.6? Aki > On 07/07/2019 18:12 Arnold Opio Oree via dovecot < > dovecot(a)dovecot.org > > > wrote: > > > Dovecot Team, > > I'd like to report a number of bugs, that are to my view all > critical. > > System: Replicated on multiple Debian 10 (Buster) systems > Dovecot Version(s): 2.3.4.1 > > doveadm-sync -1/general > > 1) If DIRNAMEs are not different between command line and > mail_location doveadm sync will fail, saying that the source and > destination directories are the same > > 2) The -n / -N flags do not work, and a sync will fail strangely if > location is specified in the namespace definition > > 3) Adds mbox to path name under mailbox directory (where syncing from > an mbox source) > > 4) Not having the mailboxes at source named the same as those at > destination causes errors and partial sync > > 5) Not having the target mailboxes formatted to receive the sync > (/<mailboxroot>/DIRNAME/) will cause sync errors. > > doveadm-sync > > 1) With large synchronizations UIDs are corrupted where multiple > syncs are executed and the program can no longer synchronize > > dovecot > > 1) Panics and fails to expand ~ to user home: observed cases are > where multiple namespaces are being used > > Please let me know if you need me to elaborate or to provide any > further information that you may need to replicate the bugs, or if I > can help in any other way. > > With regards to the last error that I requested help on i.e. > \Noselect. This has been resolved more-or-less by the workarounds > that I have implemented for the bugs reported above. > > I have seen a number of threads whilst researching the \Noselect > issue where people have been very confused. My finding was that > \Noselect is a function of the IMAP specification server-side > implementation RFC3501 ( > https://tools.ietf.org/html/rfc3501#section-6.3.6 > ). And for me the > server was returning directories with \Noselect because the mailboxes > were malformed on account of dovadm-sync errors. In order to fix this > I formed a bash command to transverse the mailbox hierarchy and > create the missing folders critical to the sdbox format, namely > DIRNAME. > > Kind regards, > > Arnold Opio Oree > Chief Executive Officer > Parallax Digital Technologies > > arnoldoree(a)parallaxdt.com > > > http://www.parallaxdt.com > > > tel : +44 (0) 333 577 8587 > fax : +44 (0) 20 8711 2477 > > Parallax Digital Technologies is a trading name of Parallax Global > Limited. U.K. Co. No. 08836288 > > The contents of this e-mail are confidential. If you are not the > intended recipient you are to delete this e-mail immediately, > disregard its contents and disclose them to no other persons. > > -----Original Message----- > From: Arnold Opio Oree via dovecot < > dovecot(a)dovecot.org > > > Reply-To: > arnoldoree(a)parallaxict.com > , Arnold Opio Oree < > arnold.oree(a)parallaxict.com > > > To: > dovecot(a)dovecot.org > > Cc: > rs(a)sys4.de > , > aki.tuomi(a)open-xchange.com > > Subject: Re: Applying Dovecot for a large / deep folder-hierarchy > archive. > Date: Thu, 04 Jul 2019 14:52:28 +0100 > > > Hi all, > > The guidance provided so far has been really helpful, and has helped > a great deal to bringing down wasted energy on finding and executing > a viable path. I am now at the final due action to complete our > Dovecot application to our use-case, but am stuck on an issue that I > cannot find any easily accessible documentation on. > > Generally this is what has been done: > > 1. Uploaded the enterprise data PST to the target groupware server. > 2. Prepared the server by changing the mailbox format to sdbox and > the the Dovecot mail location to > mail_location=/var/vmail/domain/user/mail/ > 3. Converted the pst (on-server) to a recursive mbox hierarchy using > readpst > 4. Executed doveadm-sync to convert mbox hierarchy data into sdbox > and to copy it into the enterprise archive user's mailboxes > 4.i. The biggest issue I faced at this point was doveadm-sync saying > that the source and destination pointed to the same location, whereas > they clearly did not. > 4.i.a. I resolved this by removing the location= setting from the > target namespace, and allowing it to default to mail_location = > setting, and then using a completely different DIRNAME for the import > doveadm-sync execution (which was the desired final DIRNAME); I then > once the sync had been successful, changed the mail_location DIRNAME > so that it pointed to the imported mail DIRNAME; and hence the > imported email data was in the live mailboxes > 4.i.b. doveadm-import failed several times, and was throwing quite > inexplicable errors, so I moved onto doveadm-sync > 4.i.c. I also had to make sure that the source and destination folder > names matched, otherwise doveadm-syc threw very many errors and only > partially imported the data > 4.i.d. An issue which I decided just to live with is that an mbox > DIRNAME was added to each mailbox as well as the DIRNAME specified so > the path to mail is mbox/dbox-Mails. My thought is that with the data > live on an IMAP server it will be possible to do a dysync through TCP > to correct this problem. > > The final issue that I am facing now, is that when readpst finds > empty folders in the source pst hierarchy, it does not create an mbox > file in the mbox hierarchy folder space. This causes doveadm-sync to > not create the target data required for its mailbox structure i.e. > DIRNAME sub-folder and index file (with our configuration). At this > point either doveadm-sync or the dovecot process makes these empty > folders not selectable. > > The question now is how would I go about making all of these folders > selectable, e.g. with an internal or external command line tool to > change flags / create necessary sdbox mailbox constituent data? > > Many thanks, > > Arnold Opio Oree > Chief Executive Officer > Parallax Digital Technologies > > arnoldoree(a)parallaxdt.com > > > http://www.parallaxdt.com > > > tel : +44 (0) 333 577 8587 > fax : +44 (0) 20 8711 2477 > > Parallax Digital Technologies is a trading name of Parallax Global > Limited. U.K. Co. No. 08836288 > > The contents of this e-mail are confidential. If you are not the > intended recipient you are to delete this e-mail immediately, > disregard its contents and disclose them to no other persons. > > -----Original Message----- > From: Arnold Opio Oree via dovecot < > dovecot(a)dovecot.org > > > Reply-To: > arnoldoree(a)parallaxict.com > , Arnold Opio Oree < > arnold.oree(a)parallaxict.com > > > To: Robert Schetterer < > rs(a)sys4.de > >, > dovecot(a)dovecot.org > > Subject: Re: Applying Dovecot for a large / deep folder-hierarchy > archive. > Date: Thu, 27 Jun 2019 12:05:35 +0100 > > > Also you may run into client limits.... i ve seen this with > > outlook, > > apple mail, thunderbird via imap in the past > > Thanks for this note Robert, it was not really an aspect that I was > considering. > > We are operating our groupware services user access through both > Evolution Groupware and KDE Kontact / KMail on Debian Linux > workstations. Hopefully if there is a client issue it should be local > to only one groupware client. > > I will be sure to study / investigate in this - client - area should > any issues that are not traceable to the server-side arise. > > Many thanks, > > Arnold Opio Oree > Chief Executive Officer > Parallax Digital Technologies > > arnoldoree(a)parallaxdt.com > > > > http://www.parallaxdt.com > > > > tel : +44 (0) 333 577 8587 > fax : +44 (0) 20 8711 2477 > > Parallax Digital Technologies is a trading name of Parallax Global > Limited. U.K. Co. No. 08836288 > > The contents of this e-mail are confidential. If you are not the > intended recipient you are to delete this e-mail immediately, > disregard > its contents and disclose them to no other persons. > > > -----Original Message----- > From: Robert Schetterer via dovecot < > dovecot(a)dovecot.org > > > Reply-To: Robert Schetterer < > rs(a)sys4.de > > > To: > dovecot(a)dovecot.org > > > Subject: Re: Applying Dovecot for a large / deep folder-hierarchy > archive. > Date: Thu, 27 Jun 2019 12:53:49 +0200 > > Am 27.06.2019 um 07:35 schrieb Aki Tuomi via dovecot: > > On 26.6.2019 22.12, Arnold Opio Oree via dovecot wrote: > > > Hello to you all, > > > > > > I'd like to ask about my intended application of Dovecot to > > > create > > > a folder-hierarchy for storing our enterprise emails, which are > > > treated as live data rather than archives for compliance or > > > occasional / reactive retrieval. > > > > > > The data is presently not that large (a few gigabytes), but it is > > > expected to grow rapidly. Up to this stage the data has been > > > contained > > > in a Microsoft Exchange mailbox (2013), and then in an offline > > > PST. > > > The move to the offline PST was by necessity, as the large number > > > of > > > folders, and depth of hierarchy to my best understanding caused > > > the > > > exchange server / outlook / evolution mail clients to begin to > > > malfunction. To cope with this the archive was broken up and the > > > bulk stored in the offline PST and the most active components > > > stored in > > > online Exchange mailboxes. > > > > > > I have some understanding of the fs mbox format, and also the > > > mitigations to be made for certain characters / strings. My main > > > concern is whether Dovecot is likely to be able to cope well with > > > a > > > large number of folders / depth of hierarcy. > > > > > > I will really appreciate any help / advice you can give. > > > > > > Best regards, > > > > > > Arnold Opio Oree > > > > > > > Hi! > > > > Dovecot 2.2.34/2.3 supports unlimited folder depth, the only > > limiting > > factor is that the total name may not exceed 4096 bytes. Also > > individual > > folder names may not exceed 255 bytes. > > > > Prior to that the limit is 255 per folder up to 16 levels. > > > > I can't recommend using 'mbox' storage format, please consider > > using > > maildir or sdbox instead. > > > > Aki > > > > > > > > > > --- Aki Tuomi

2 2

Re: Applying Dovecot for a large / deep folder-hierarchy archive - BUG REPORTS!
by Adam D. Barratt 10 Jul '19

10 Jul '19

On 2019-07-10 12:01, Arnold Opio Oree wrote: Please stop CCing this thread to the Debian Release Management mailing list. If there are issues with the Dovecot packages in Debian then they should be reported as bugs against the packages. Thanks. Adam (Debian Release Team member)

2 1

Error: last_login_dict: Failed to write value: dict-server returned failure: sql dict: commit failed: Field 'password' doesn't have a default value
by Lefteris Tsintjelis 10 Jul '19

10 Jul '19

Hi, after upgrading my database I kept on getting those errors. It seems like later versions of databases have strict mode enabled by default and it must either be disabled or change the dovecot database schema. Besides the password, would anyone know what other fields must be changed to the dovecot schema and have a default value? Is there any other dovecot schema for strict mode? Lefteris

1 0

Applying Dovecot for a large / deep folder-hierarchy archive.
by Arnold Opio Oree 10 Jul '19

10 Jul '19

Hello to you all, I'd like to ask about my intended application of Dovecot to create a folder-hierarchy for storing our enterprise emails, which are treated as live data rather than archives for compliance or occasional / reactive retrieval. The data is presently not that large (a few gigabytes), but it is expected to grow rapidly. Up to this stage the data has been containedin a Microsoft Exchange mailbox (2013), and then in an offline PST. The move to the offline PST was by necessity, as the large number offolders, and depth of hierarchy to my best understanding caused the exchange server / outlook / evolution mail clients to begin tomalfunction. To cope with this the archive was broken up and the bulk stored in the offline PST and the most active components stored inonline Exchange mailboxes. I have some understanding of the fs mbox format, and also the mitigations to be made for certain characters / strings. My main concern is whether Dovecot is likely to be able to cope well with a large number of folders / depth of hierarcy. I will really appreciate any help / advice you can give. Best regards, Arnold Opio OreeChief Executive OfficerParallax Digital Technologies arnoldoree(a)parallaxdt.com http://www.parallaxdt.com tel : +44 (0) 333 577 8587fax : +44 (0) 20 8711 2477 Parallax Digital Technologies78A Hatfeild MeadLondonSM4 5PF Parallax Digital Technologies is a trading name of Parallax GlobalLimited. U.K. Co. No. 08836288 The contents of this e-mail are confidential. If you are not theintended recipient you are to delete this e-mail immediately, disregardits contents and disclose them to no other persons.

5 11