November 2018 - dovecot

RSA+ECC certificates
by Alex Hautequest 17 Nov '18

17 Nov '18

Any way of using both a RSA and an ECC certificates, letting the client pick the one to use? Alex

2 1

Mail slowed down to a crawl...
by StarionTech (IMAP) 16 Nov '18

16 Nov '18

Been moving along just fine for a couple years now, then in the last two days, email has slowed to a crawl. Retrieving email via IMAP is very slow, progress bar in the mail client shows that it is downloading messages almost constantly, like it never closes the connection, or it’s getting the messages very slowly. The server that dovecot/postfix is on has also bogged down to a crawl. The postfix queue is sitting with hundreds of queued up good customer email. It’s almost acting like dovecot is not getting the messages from postfix, or they’re being delayed for some reason. How to approach fixing this? Jeff J.

4 4

huge increase in storage activity afther dovecot upgrade
by Adrian M 16 Nov '18

16 Nov '18

Hi, we upgraded our servers from version 2.2.13 to 2.2.27. After the upgrade we notice a 10x increase in traffic with the nfs storage an errors like this in the logfile: Nov 12 09:48:16 mail dovecot: imap(...): Error: Corrupted index cache file /.../dovecot.index.cache: invalid record size Nov 12 09:48:16 mail dovecot: imap(...): Error: unlink(/.../dovecot.index.cache) failed: No such file or directory (in mail-cache.c:29) Nov 12 09:48:16 mail dovecot: imap(...): Error: Corrupted index cache file /.../dovecot.index.cache: invalid record size Nov 12 09:48:16 mail dovecot: imap(...): Error: Broken file /.../dovecot-uidlist line 8: Invalid data: Is this normal ? Will the activity wind down ? Can we do something, like deleting the old dovecot.index* or dovecot-uidlist files from maildirs, or doing an doveadm force-refresh for all inboxes ? Thank you ! Here's my configuration: # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6 nfs auth_failure_delay = 15 secs auth_mechanisms = plain login auth_verbose = yes auth_worker_max_count = 256 default_client_limit = 40000 default_process_limit = 512 dict { lastlogin = mysql:/etc/dovecot/mysql/dovecot-dict-lastlogin.conf quotadict = mysql:/etc/dovecot/mysql/dovecot-dict-quota.conf } disable_plaintext_auth = no first_valid_uid = 100 log_timestamp = "%Y-%m-%d %H:%M:%S " login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pid=%p %c mail_fsync = always mail_location = maildir:*/home/virtual/*%d/%u mail_max_userip_connections = 16 mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_log notify mail_privileged_group = mail mailbox_list_index = yes maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox virtual/All { special_use = \All } prefix = } passdb { args = /etc/dovecot/mysql/dovecot-sql.conf driver = sql } plugin { last_login_dict = proxy::lastlogin last_login_key = last-login/%u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:user::proxy::quotadict quota_rule2 = Trash:ignore quota_rule3 = Spam:ignore quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 The email account that you tried to reach is over quota quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes trash = /etc/dovecot/dovecot-trash.conf zlib_save = gz zlib_save_level = 6 } protocols = imap sieve pop3 lmtp service anvil { client_limit = 20000 unix_listener anvil-auth-penalty { mode = 00 } } service auth { client_limit = 20000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = mail mode = 0660 user = vmail } user = root vsz_limit = 1 G } service dict { unix_listener dict { mode = 0660 user = vmail } } service imap-login { client_limit = 4096 executable = /usr/lib/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 4096 process_min_avail = 24 service_count = 100 } service imap { executable = /usr/lib/dovecot/imap process_limit = 4096 process_min_avail = 24 vsz_limit = 1 G } service lmtp { process_min_avail = 24 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } vsz_limit = 1 G } service managesieve-login { client_limit = 4096 inet_listener sieve { port = 4190 } process_limit = 4096 process_min_avail = 24 service_count = 1 } service managesieve { process_limit = 4096 vsz_limit = 1 G } service pop3-login { client_limit = 4096 executable = /usr/lib/dovecot/pop3-login inet_listener pop3 { address = * port = 0 } inet_listener pop3s { address = * port = 995 } process_limit = 4096 process_min_avail = 24 service_count = 100 } service pop3 { executable = /usr/lib/dovecot/pop3 process_limit = 4096 process_min_avail = 24 vsz_limit = 1 G } service quota-status { client_limit = 4096 executable = /usr/lib/dovecot/quota-status -p postfix process_min_avail = 24 unix_listener /var/spool/postfix/private/quota-status { group = postfix mode = 0660 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0660 user = vmail } user = vmail } userdb { args = /etc/dovecot/mysql/dovecot-sql.conf driver = sql } protocol imap { imap_idle_notify_interval = 4 mins imap_max_line_length = 64 k mail_max_userip_connections = 16 mail_plugins = zlib quota mail_log notify trash imap_quota } protocol pop3 { mail_max_userip_connections = 16 mail_plugins = zlib quota mail_log notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_no_flag_updates = yes pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = zlib quota mail_log notify sieve sendmail_path = /usr/lib/sendmail syslog_facility = mail } protocol sieve { mail_max_userip_connections = 16 managesieve_max_compile_errors = 5 } protocol lmtp { mail_plugins = zlib quota mail_log notify sieve syslog_facility = mail }

4 8

failed: Invalid cross-device link
by KES 15 Nov '18

15 Nov '18

Hi. I mount ~/Maildir/new via s3fs s3fs mybucket /home/kes/Maildir/new -f -o use_cache=/tmp/S3 -o noatime -o endpoint=eu-west-1 -o profile=mail -o allow_other -o uid=1000,gid=1000 I configure dovecot: maildir_copy_with_hardlinks = no Nov 15 19:30:48 work dovecot: imap(kes): Error: rename(/home/kes/Maildir/new/ddd3, /home/kes/Maildir/cur/ddd3:2,) failed: Invalid cross-device link Tell me please, why I still get this error and how to fix it?

2 1

dovecot 2.2/openssl 1.0 vs dovecot 2.3/openssl 1.1.1 ssl regression
by Arkadiusz Miśkiewicz 15 Nov '18

15 Nov '18

Hi. I'm considering dovecot migration from 2.2.36 run with openssl 1.0.2o to dovecot 2.3.3 run with openssl 1.1.1. Currently I have both variants running with identical configs and certs (the only differences are due to config syntax changes in dovecot 2.3), so for example on both I have: ssl_ca = </etc/openssl/certs/wildcard_ca.pem (this file contains single intermediate certificate of my CA) ssl_cert = </etc/openssl/certs/wildcard_crt.pem (this contains single cerificate for my *.example.com domain) ssl_key = # hidden, use -P to show it (and one key) No alt certs in use. Chain is: - CA trusted by clients (this certificate isn't provided by my dovecot, it's not needed) - wildcard_ca.pem - intermediate CA - wildcard_crt.pem - wildcard certificate for my *.example.com domain dovecot 2.2.36 behaviour is to provide wildcard_ca.pem and wildcard_crt.pem to the client - that behaviour is OK. Client has full trust chain. dovecot 2.3.3 provides only wildcard_crt.pem certificate to the client which is a big problem because missing wildcard_ca.pem (intermediate certificate) breaks chain and client is not able to verify trust chain. Testing is done with simple: openssl s_client -connect my.example.com:143 -starttls imap -servername my.example.com -showcerts 2.3.x announcements and upgrade wiki mention no such behaviour change, so I assume it is a regression. Now doing cat wildcard_ca.pem >> wildcard_crt.pem solves the problem and dovecot starts providing both certs to clients but if that's the proper way of solving this issue then what's the point of having ssl_ca config setting? Ideas? -- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

3 3

FETCH BODY vs FETCH BODYSTRUCTURE + FETCH BODY.PEEK
by Alessio Cecchi 15 Nov '18

15 Nov '18

Hello, we are developing a web page, for a ticket system, to show the email messages ordered by the biggest attachments size present in an IMAP folder, with a pagination of 100 messages per page. Each raw represents a message with "DATE, FROM, SUBJECT, the preview (the first 500 chars of the text/html of the messsage) and the list of the attachments (name + size) it contains. The problem is that the page load is VERY slow because we fetch the entire BODY of all 100 messages, in order to get the preview and the attachments name and size. Now, considering that each messages can contain up to 50 MB of attachments, we risk to download 5000MB from IMAP for every page. An option might be to perform two FETCH for each message: - one fetch with just the BODYSTRUCTURE (where we can get the attachments name and size and the text/html parts). - one fetch with the body parts (BODY.PEEK[]) we need to build the preview (text + html). We are wondering if there is a single FETCH command to get the text and html parts (needed to build the preview) with only the name and the size of the attachments without downloading the entire message. Thanks. -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 1

Dovecot proxy: per user/domain 'namespace/inbox/prefix' from MySQL
by Adi Pircalabu 15 Nov '18

15 Nov '18

As a way to try and avoid using "prefix = INBOX." ad infinitum for the inbox namespace, I'm looking for ways to move on to "prefix =" for new mail accounts, and grandfather the existing ones. Previously running Courier-IMAP, now Dovecot, I looked at https://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_Courier_IMAP and decided it's too risky to go down that path and use namespace compat, with so many IMAP clients out there the scope of testing is huge and the outcome is uncertain and not worth it. After reading https://wiki.dovecot.org/Namespaces#Per-user_Namespace_Location_From_SQL I thought I might be able to overwrite the server configuration per user returning 'namespace/inbox/prefix' value from SQL. Here's the setup I attempted, briefly: 1. Client connects to the Dovecot proxy, which authenticates the user and proxies to the backend using a query like this in /etc/dovecot/conf.d/dovecot-sql.conf.ext: driver = mysql connect = <connection_string> password_query = SELECT NULL AS password, 'Y' as nopassword, host, 'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox WHERE email = '%u' AND disabled_smtpauth=0 Works a treat. 2. Next, I'm trying to add the prefix lookup in the picture. In the same file I've added: user_query = SELECT ns_inbox_prefix AS 'namespace/inbox/prefix' FROM mailbox WHERE email = '%u' AND disabled_smtpauth=0 3. The mailbox table schema reads: CREATE TABLE `mailbox` ( `id` int(11) unsigned NOT NULL AUTO_INCREMENT, `email` varchar(255) NOT NULL DEFAULT '', `password` varchar(255) NOT NULL DEFAULT '', `clear_password` varchar(255) NOT NULL DEFAULT '', `name` varchar(255) NOT NULL DEFAULT '', `host` varchar(32) DEFAULT NULL, `port` varchar(32) DEFAULT NULL, `ns_inbox_prefix` varchar(255) NOT NULL DEFAULT '', `lastlog_remote_ips` bigint(20) unsigned NOT NULL DEFAULT 0, `curlog_remote_ips` bigint(20) unsigned NOT NULL DEFAULT 0, `disabled_smtpauth` tinyint(1) NOT NULL DEFAULT 0, `last_modified` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(), PRIMARY KEY (`id`), UNIQUE KEY `email` (`email`) ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1; The 2 queries above return: MariaDB [postfix]> SELECT NULL AS password, 'Y' as nopassword, host, 'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox WHERE email = 'adi2(a)0aditest.local' AND disabled_smtpauth=0; +----------+------------+----------------+----------+-------+ | password | nopassword | host | starttls | proxy | +----------+------------+----------------+----------+-------+ | NULL | Y | 192.168.123.24 | any-cert | Y | +----------+------------+----------------+----------+-------+ 1 row in set (0.00 sec) MariaDB [postfix]> SELECT ns_inbox_prefix AS 'namespace/inbox/prefix' FROM mailbox WHERE email = 'adi2(a)0aditest.local' AND disabled_smtpauth=0; +------------------------+ | namespace/inbox/prefix | +------------------------+ | | +------------------------+ 1 row in set (0.00 sec) After reloading dovecot service with auth_debug = yes are the maillog for an IMAP session: Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Nov 15 12:43:48 proxy1 dovecot: auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Nov 15 12:43:48 proxy1 dovecot: auth: Debug: auth client connected (pid=7527) Nov 15 12:43:53 proxy1 dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011session=AorrLqp6drgAAAAAAAAAAAAAAAAAAAAB#011lip=::1#011rip=::1#011lport=1143#011rport=47222#011resp=<hidden> Nov 15 12:43:53 proxy1 dovecot: auth: Debug: sql(adi2@0aditest.local,::1,<AorrLqp6drgAAAAAAAAAAAAAAAAAAAAB>): cache miss Nov 15 12:43:53 proxy1 dovecot: auth-worker(7533): Debug: Loading modules from directory: /usr/lib64/dovecot/auth Nov 15 12:43:53 proxy1 dovecot: auth-worker(7533): Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.so Nov 15 12:43:53 proxy1 dovecot: auth-worker(7533): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_mysql.so Nov 15 12:43:53 proxy1 dovecot: auth-worker(7533): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Nov 15 12:43:53 proxy1 dovecot: auth-worker(7533): Debug: sql(adi2@0aditest.local,::1,<AorrLqp6drgAAAAAAAAAAAAAAAAAAAAB>): query: SELECT NULL AS password, 'Y' as nopassword, host, 'any-cert' as 'starttls', 'Y' AS proxy FROM mailbox WHERE email = 'adi2(a)0aditest.local' AND disabled_smtpauth=0 Nov 15 12:43:53 proxy1 dovecot: auth: Debug: client passdb out: OK#0111#011user=adi2@0aditest.local#011host=192.168.123.24#011starttls=any-cert#011proxy#011pass=<hidden> Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...] Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...] Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...] Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...] Nov 15 12:43:53 proxy1 dovecot: imap-login: Invalid certificate: [...] Nov 15 12:43:53 proxy1 dovecot: imap-login: proxy(adi2(a)0aditest.local): started proxying to 192.168.123.24:143: user=<adi2(a)0aditest.local>, method=PLAIN, rip=::1, lip=::1, secured, session=<AorrLqp6drgAAAAAAAAAAAAAAAAAAAAB> Looks like user_query isn't executed, why? And here's the corresponding IMAP session: Trying ::1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. . LOGIN adi2(a)0aditest.local <hidden> . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE NOTIFY SPECIAL-USE QUOTA] Logged in . NAMESPACE * NAMESPACE (("INBOX." ".")) NIL NIL . OK Namespace completed (0.000 + 0.000 secs). . LIST "" * * LIST (\HasChildren) "." INBOX * LIST (\HasNoChildren \Trash) "." INBOX.Trash * LIST (\HasNoChildren) "." INBOX.Templates * LIST (\HasNoChildren \Sent) "." INBOX.Sent * LIST (\HasNoChildren \Drafts) "." INBOX.Drafts * LIST (\HasNoChildren \Archive) "." INBOX.Archives * LIST (\HasNoChildren \UnMarked \Junk) "." INBOX.Spam . OK List completed (0.000 + 0.000 secs). . LSUB "" * * LSUB (\Archive) "." INBOX.Archives * LSUB (\Drafts) "." INBOX.Drafts * LSUB (\Sent) "." INBOX.Sent * LSUB (\Junk) "." INBOX.Spam * LSUB () "." INBOX.Templates * LSUB (\Trash) "." INBOX.Trash . OK Lsub completed (0.000 + 0.000 secs). . LOGOUT * BYE Logging out . OK Logout completed (0.000 + 0.000 secs). Connection closed by foreign host. How do I overwrite 'namespace/inbox/prefix' for an user on the Dovecot proxy? Is user_query working in this context? -- Adi Pircalabu

1 1

different TLS protocols on different ports
by A. Schulze 15 Nov '18

15 Nov '18

Hello, I'm providing IMAP+Starttls on port 143 for users with legacy MUA. So I've to enable TLS1.0 up to TLS1.3 For IMAPS / port 993 I like to enable TLS1.2 and TLS1.3 only. Is this possible with dovecot-2.2.36 / how to setup this? Thanks for suggestions, Andreas

6 9

Re: Disaster Recovery Help
by Giovanni Bisanti 14 Nov '18

14 Nov '18

I just tried, and no, it does not get email addresses. and the user only has like 12 months on the device, anything older is only headers. It's so bad!! Thanks for everybody's help Giovanni Bisanti helpdesk(a)techtoyoullc.com (mailto:helpdesk@techtoyoullc.com) (239) 851-7222 November 12, 2018 10:16 PM, "James Brown" wrote: Don’t know. Have only used it for Messages. James. On 13 Nov 2018, at 10:07 am, Giovanni Bisanti wrote: Yes, I think I have to try this. Have you ever used iMazing? does it export emails? On Tue, 2018-11-13 at 10:05 +1100, James Brown wrote: On 13 Nov 2018, at 9:48 am, Giovanni Bisanti wrote: the Account on the iPhone is IMAP, I'm not sure I understand the process you are describing iMazing might be able to help you get the mail off the iPhone: https://imazing.com (https://imazing.com/) Then transfer to you Dovecot Maildir. (Have never done this - it’s just a suggestion). James.

1 0

autoexpunged in IMAP logout format
by Alessio Cecchi 14 Nov '18

14 Nov '18

Hi, in my imap_logout_format I register %{deleted} %{expunged} %{trashed} but now I saw that is available also %{autoexpunged}. Is necessary to add also %{autoexpunged} in logout format to have the count of all deleted email via IMAP? Thanks -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

1 0