October 2019 - dovecot

Recovering emails from "dead" server
by Mark | Small World Systems 29 Oct '19

29 Oct '19

I have a Linix server that won't boot. I've patched in a live CD and mounted the original HDDs so I can browse the files using SSH. I am trying to find where all my customer emails are stored so that I can (hopefully) copy them off this "dead" server onto a new server which is also running cPanel and Dovecot. Does anyone know where Dovecot stores emails for each email account when it runs under cPanel? I can see a directory called /home/<cpanel username>/mail/<domain name>/<mail username> (I've used <xxxxx> in place of the actual text to obviscate the actual user information). If I navigate to that directory I can see lots of interesting sub directories e.g. cur, new and tmp plus some files such as: dovecot.index.cache, dovecote.index, dovecote-uidlist But then I'm stumped. I was hoping to find a big fat directory that contained all the mail files. Is anyone able to tell me what I need to copy across to get all the old emails off the "dead" server onto the new one? Thank you.

3 2

Sieve problem - Timeout while waiting for lock
by Domenico Pastore 29 Oct '19

29 Oct '19

Hi, I have an issue with dovecot 2.3.7.2. When lmtp stores a message into INBOX, I get this error: 2019-10-29 16:32:53 lmtp(“test@XXXX”)<28874><4IPiEytbuF3KcAAAioI89A>: Error: sieve: Execution of script /config/dovecot/common/default.sieve was aborted due to temporary failure 2019-10-29 16:32:53 lmtp(“test@XXXX”)<23470>: Error: sieve: msgid=CE595339-2976-6DFD-FDAD-EEA15CC53218@XXXX: failed to store into mailbox ‘INBOX’: Timeout while waiting for lock This problem only occurs for one mailbox and when it receives a big amount of messages. With the previous version (dovecot 2.2.15) this did never occur. The file /config/dovecot/common/default.sieve is in local server, no on NFS filesystem BR, Domenico ----------------- # 2.3.7.2 (3c910f64b): /config/dovecot/dovecot/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-1062.1.2.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core) auth_debug = yes auth_master_user_separator = * auth_username_format = base_dir = /data/dovecot/var/run/dovecot default_client_limit = 8192 default_vsz_limit = 768 M dict { sqlquota = mysql:/config/dovecot/common/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_gid = 101 first_valid_uid = 102 imap_id_send = import_environment = TZ info_log_path = /LOGS/dovecot/dovecot.log instance_name = dovecot listen = dovecot log_path = /LOGS/dovecot/dovecot.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_gid = 101 mail_location = maildir:%h/Maildir mail_max_userip_connections = 50 mail_plugins = quota fts fts_solr mail_log notify virtual mail_uid = 102 maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug mbox_write_locks = fcntl namespace { hidden = yes list = no location = virtual:/config/dovecot/common/virtual:INDEX=~/virtual:SUBSCRIPTIONS=.subscriptions/%u prefix = Messaggi in arrivo. separator = . subscriptions = no } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = /config/dovecot/common/passwd.masterusers driver = passwd-file master = yes } passdb { args = /config/dovecot/dovecot/dovecot-ldap.conf driver = ldap } passdb { args = /config/dovecot/common/dovecot-ldap-multiuser.conf driver = ldap master = yes } passdb { args = /config/dovecot/common/dovecot-ldap-webmail.conf.ext driver = ldap master = yes } plugin { fts = solr fts_autoindex = yes fts_autoindex_max_recent = 1000 fts_solr = url=https://192.160.10.81:8983/solr/dovecot/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags quota = dict:User quota::proxy::sqlquota quota_rule2 = INBOX.Trash:storage=+100M sieve_default = /dovecot/common/default.sieve sieve_extensions = +vnd.dovecot.debug } postmaster_address = posta(a)XXXX.it service anvil { client_limit = 8000 } service auth { client_limit = 12000 unix_listener auth-userdb { mode = 0600 } } service dict { process_limit = 7500 process_min_avail = 10 unix_listener dict { mode = 0600 user = dovecot } } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 inet_listener { port = 924 } privileged_group = process_limit = 0 process_min_avail = 10 protocol = service_count = 100 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 5000 process_min_avail = 10 } service imap { drop_priv_before_exec = yes process_limit = 5000 process_min_avail = 10 } service lmtp { inet_listener lmtp { port = 24 } process_limit = 2500 process_min_avail = 10 } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { drop_priv_before_exec = yes process_limit = 1024 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 1000 process_min_avail = 5 } service pop3 { drop_priv_before_exec = yes process_limit = 1000 process_min_avail = 5 } ssl_cert = </config/dovecot/common/certs/dovecot.pem ssl_client_ca_file = /config/solr-certs/solr-ssl.pem ssl_key = # hidden, use -P to show it userdb { args = /config/dovecot/dovecot/dovecot-ldap.conf driver = ldap } protocol lmtp { mail_plugins = quota mail_log notify sieve virtual fts fts_solr } protocol lda { mail_plugins = quota mail_log notify sieve virtual } protocol imap { mail_plugins = quota imap_quota mail_log notify virtual fts fts_solr } protocol pop3 { mail_plugins = quota mail_log notify virtual pop3_uidl_format = UID%u-%v }

1 1

Error: Mailbox INBOX: Broken file dovecot-uidlist: Invalid data
by Dirk 29 Oct '19

29 Oct '19

Hi list, today a user reported messages are disappearing from his INBOX and I discovered a two digit number of mailboxes with broken dovecot-uidlist files. The log says: Oct 29 08:51:34 mail02 dovecot: lmtp(info(a)example.com)<5270><QOoVJIbvt12WFAAAJOVBUw>: Error: Mailbox INBOX: Broken file /var/vmail/example.com/info/dovecot-uidlist line 88: Invalid data: And the respective files do look broken: 4596 :1572344253.M529133P14861.mail03.example.com,S=53053,W=54233 ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@4597 :1572344797.M93833P15905.mail03.example.com,S=126866,W=129256 Setup : Behind a director I am running two dovecot backend servers sharing the vmail directory via NFS. Do you have an idea where this might come from? How can I debug this further? Cheers, Dirk

1 0

changing cipher for imap clients
by Fourhundred Thecat 29 Oct '19

29 Oct '19

When my client connects, I see this in my log: dovecot: imap-login: TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits) Whereas, when client connects to my postfix server, I see: Anonymous TLS connection established from * TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits) how can I tell dovecot to use AES256, instead of AES128 ? is this set by ssl_cipher_list ? Here are my current values (defaults) # doveconf ssl_cipher_list ssl_cipher_list = ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH # dovecot --version 2.3.4.1 thanks,

4 4

Dovecot and MySQL aborted connections.
by Reio Remma 28 Oct '19

28 Oct '19

Is anyone else using Dovecot (2.3.8) with MySQL (5.7) seeing a lot of these in MySQL logs? 2019-10-28T11:08:20.384428+02:00 58378 [Note] Aborted connection 58378 to db: 'vmail' user: 'vmail' host: 'localhost' (Got an error reading communication packets) 2019-10-28T11:10:09.821171+02:00 58420 [Note] Aborted connection 58420 to db: 'vmail' user: 'vmail' host: 'localhost' (Got an error reading communication packets) 2019-10-28T11:11:26.170015+02:00 58441 [Note] Aborted connection 58441 to db: 'vmail' user: 'vmail' host: 'localhost' (Got an error reading communication packets) 2019-10-28T11:13:14.091426+02:00 58459 [Note] Aborted connection 58459 to db: 'vmail' user: 'vmail' host: 'localhost' (Got an error reading communication packets) They've plagued my logs for as long as I can remember. Is Dovecot not closing connections to the database properly or something similar? Reio

3 3

Dovecot list breaks DKIM.
by Reio Remma 28 Oct '19

28 Oct '19

Hello again, I noticed mails from the Dovecot list break DKIM signatures. Perhaps it's something to look at? Most lists I'm on manage to pass messages with DKIM intact. Authentication-Results: abc.abc.abc; dkim=fail (rsa verify failed) header.d=mrstuudio.ee header.s=mr header.b=M03Fp5lE; dmarc=pass (policy=none) header.from=dovecot.org; spf=pass Thanks, Reio

1 0

SQL iterate_query
by Daniel Miller 28 Oct '19

28 Oct '19

I've been hunting some ghost mailboxes - and I *think* I found the source. I use the complete email address as the username, and store such in a database. The storage structure is location=/var/mail/%d/%n. Not unusual I think. So all I *should* see from "ls /var/mail" would be a list of domains. But I keep seeing empty mailboxes being created at this level. Having corrected a few other errors I *hope* I've found the last one - but if I'm right I believe the docs need updating: The examples given for SQL userdb's include: iterate_query = SELECT userid AS username, domain FROM users So this means the username is returned for *both* the username and domain. Even if I'm wrong as to the cause of my own troubles this can't be right. It just can't. Or am I mistaken? So, given that the complete address is used as the username I now use: iterate_query = SELECT username FROM mailbox (I'm using postfixadmin to administer this - and "mailbox" is the default user table name) I believe the alternative would be an explicit: iterate_query = SELECT username, domain AS username, domain FROM users I don't *think* that would make any security difference for my use case so why add the extra processing? I believe the documentation should be updated, or at least clarified, on this issue. -- Daniel

2 1

Configure to only receive recent mail?
by lty 28 Oct '19

28 Oct '19

I am looking for imap or pop3 to receive only the most recent emails. Virtual-plugin seems to meet my needs https://doc.dovecot.org/configuration_manual/virtual_plugin/#virtual-plugin > List messages from past 48 hours (syntax is in seconds): > # ~/Maildir/virtual/recent/dovecot-virtual > INBOX > work/* > all younger 172800 but this will create a virtual/*, I want to control the real INBOX. The user can set the email for how long it takes to set itself, for example, within 3 months by default.This way you can use all the mail to display to the client. Does dovecot support such a configuration?

1 0

Dovecot Index Corruption and dovecot-uidlist Issues
by Billy Luedtke 25 Oct '19

25 Oct '19

Hello, We have recently implemented dovecot directors to resolve an ongoing NFS lock/corruption issue. Everything was working great! So we updated our servers from centos 7.3 to 7.7 and dovecot 2.2.10 to 2.3.8 This was a bit of a failure and we were getting spammed with corrupt index files and logs. After we clear the index files with rm rf/mail/d/e/username/Maildir/dovecot*, they just come back in. From: dovecot-mysql-2.3.8-4.x86_64 dovecot-2.3.8-4.x86_64 dovecot-pigeonhole-2.3.8-4.x86_64 To: dovecot-2.2.10-8.el7.x86_64 dovecot-mysql-2.2.10-8.el7.x86_64 dovecot-pigeonhole-2.2.10-8.el7.x86_64 We rolled back nfs-utils and libs due to a case on redhats bug report "https://bugzilla.redhat.com/show_bug.cgi?id=1521071" libnfsidmap-0.25-19.el7.x86_64 nfs-utils-1.3.0-0.65.el7.x86_64 to libnfsidmap-0.25-15.el7.x86_64 nfs-utils-1.3.0-0.33.el7.x86_64 We rolled back our dovecot version to 2.2.10 and are still getting the following logs. Any advice would help Network view: |Dovecot Directors x 3| | |Dovecot servers x4| Storage is shared between all 4 servers | |Netapp NFSv3 Storage| Nothing changed here Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Log synchronization error at seq=89,offset=32340 for /mail/j/k/<username>/Maildir/dovecot.index: Extension record update for invalid uid=6798 Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Log synchronization error at seq=89,offset=32384 for /mail/j/k/<username>/Maildir/dovecot.index: Extension record update for invalid uid=6798 Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Synchronization corrupted index header /mail/j/k/<username>/Maildir/dovecot.index: first_recent_uid 6799 > next_uid 6798 Oct 25 01:33:25 <IP> dovecot: imap(<username>): Warning: fscking index file /mail/j/k/<username>/Maildir/dovecot.index Oct 25 01:33:25 <IP> dovecot: imap(<username>): Error: Fixed index file /mail/j/k/<username>/Maildir/dovecot.index: first_recent_uid 6799 -> 6798 Oct 25 01:33:26 <IP> dovecot: imap(<username>): Error: Broken file /mail/j/k/<username>/Maildir/dovecot-uidlist line 6461: Invalid data: Oct 25 01:33:26 <IP> dovecot: imap(<username>): Error: Log synchronization error at seq=89,offset=32488 for /mail/j/k/<username>/Maildir/dovecot.index: Append with UID 6798, but next_uid = 6799 Oct 25 01:33:26 <IP> dovecot: imap(<username>): Warning: fscking index file /mail/j/k/<username>/Maildir/dovecot.index Oct 25 01:47:14 <IP> dovecot: imap(<username>): Warning: fscking index file /mail/d/e/<username>/Maildir/dovecot.index Oct 25 01:47:14 <IP> dovecot: imap(<username>): Error: Failed to map transaction log /mail/d/e/<username>/Maildir/dovecot.index.log at sync_offset=9900 after locking: pread() failed: Stale file handle Oct 25 01:47:14 <IP> dovecot: imap(<username>): Error: Index /mail/d/e/<username>/Maildir/dovecot.index: Lost log for seq=16 offset=192: Missing middle file seq=16 (between 16..4294967295, we have seqs 14): Requested newer log than exists: Log inode is unchanged (initial_mapped=1, reason=Index mapped) Oct 25 01:51:14 <IP> dovecot: imap(<username>): Error: Broken file /mail/d/e/<username>/Maildir/dovecot-uidlist line 65: Invalid data: Dovecot -n # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-1062.1.2.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core) auth_mechanisms = plain login auth_username_format = %Ln auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no first_valid_uid = 300 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /mail/sieve/global/ sieve_global_path = /mail/sieve/default.sieve } postmaster_address = postmaster@%d protocols = imap pop3 lmtp sieve service auth { client_limit = 0 inet_listener { port = 11010 } process_limit = 1 process_min_avail = 0 service_count = 0 } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 2 process_min_avail = 2 service_count = 0 } service imap { client_limit = 1 executable = imap process_limit = 2048 process_min_avail = 50 service_count = 1 } service lmtp { client_limit = 1 process_limit = 0 process_min_avail = 0 service_count = 0 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 2 process_min_avail = 2 service_count = 0 } service pop3 { client_limit = 1 executable = pop3 process_limit = 256 process_min_avail = 25 service_count = 1 } ssl_ca = </etc/ssl/mdc/mdc.ca ssl_cert = </etc/ssl/mdc/mdc.crt ssl_cipher_list = !RC4:HIGH:MEDIUM:+TLSv1:!MD5:!SSLv2:+SSLv3:!ADH:!aNULL:!eNULL:!NULL:!DH:!ADH:!EDH:!AESGCM ssl_key = </etc/ssl/mdc/mdc.key ssl_protocols = !SSLv2 !SSLv3 userdb { args = /etc/dovecot/sql.conf.ext driver = sql override_fields = plain_pass=%w } protocol lmtp { mail_plugins = " quota sieve" } protocol imap { mail_max_userip_connections = 40 } protocol sieve { info_log_path = /var/log/dovecot-sieve.log log_path = /var/log/dovecot-sieve-errors.log managesieve_implementation_string = dovecot managesieve_max_line_length = 65536 } Billy Luedtke, Microsoft MCP System Administrator II, WIN

1 0

subscription namespace
by Daniel Miller 25 Oct '19

25 Oct '19

The current documentation makes mention of a "special" subscription namespace. The example given: namespace subscriptions { subscriptions = yes prefix = "" list = no hidden = yes } namespace inbox { inbox = yes location = subscriptions = no [...] results in a startup error as both namespaces have the same prefix. Was the intent for the "inbox" namespace to have an explicit "INBOX/" prefix? If this is configured for an existing server that previously had no such "INBOX/" prefix namespace - will clients need to be reconfigured? -- Daniel

1 0